2024-09-11T08:32:11.318 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T08:32:11.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:32:11.431 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/journal} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900665' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: daaf78de334647fc0c800f4ac495cc9475a26bae owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 6175 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 88/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: daaf78de334647fc0c800f4ac495cc9475a26bae targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRkVssnmE3WL0XhHzfwsWQgF3iOSXul2tX82pLnasxjkgPmS9sR8dNCLRSa9dGruGJngD/nX9d+m/WAPTLjIyk= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDDxr/62T/l8T1vSahPz381O0lr2vUIZokzOVktnY22s5j317G5X/d2KH0iLYvdmtfYPiBLsgUTmZu6vnLDsJ64= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - workunit: clients: client.0: - rbd/journal.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_21:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T08:32:11.431 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa; will attempt to use it 2024-09-11T08:32:11.431 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa/tasks 2024-09-11T08:32:11.432 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T08:32:11.433 INFO:teuthology.task.internal:Checking packages... 2024-09-11T08:32:11.454 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T08:32:11.454 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T08:32:11.454 INFO:teuthology.packaging:ref: None 2024-09-11T08:32:11.454 INFO:teuthology.packaging:tag: None 2024-09-11T08:32:11.454 INFO:teuthology.packaging:branch: squid 2024-09-11T08:32:11.454 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T08:32:11.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T08:32:11.658 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-308.gdaaf78de 2024-09-11T08:32:11.659 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T08:32:11.666 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T08:32:11.666 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T08:32:11.674 INFO:teuthology.task.internal:Saving configuration 2024-09-11T08:32:11.687 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T08:32:11.693 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T08:32:11.718 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 08:27:23.092046', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRkVssnmE3WL0XhHzfwsWQgF3iOSXul2tX82pLnasxjkgPmS9sR8dNCLRSa9dGruGJngD/nX9d+m/WAPTLjIyk='} 2024-09-11T08:32:11.741 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 08:27:23.090093', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDDxr/62T/l8T1vSahPz381O0lr2vUIZokzOVktnY22s5j317G5X/d2KH0iLYvdmtfYPiBLsgUTmZu6vnLDsJ64='} 2024-09-11T08:32:11.741 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T08:32:11.748 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-11T08:32:11.748 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-11T08:32:11.748 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T08:32:11.810 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1b52a87250>, signals=[15]) 2024-09-11T08:32:11.810 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T08:32:11.816 INFO:teuthology.task.internal:Opening connections... 2024-09-11T08:32:11.816 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:32:11.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:32:11.893 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:32:11.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:32:11.967 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T08:32:11.974 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-09-11T08:32:11.992 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-09-11T08:32:11.992 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-09-11T08:32:12.049 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-09-11T08:32:12.049 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-09-11T08:32:12.049 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-09-11T08:32:12.049 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-09-11T08:32:12.049 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T08:32:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T08:32:12.051 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-09-11T08:32:12.071 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-09-11T08:32:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-09-11T08:32:12.090 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-09-11T08:32:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-09-11T08:32:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-09-11T08:32:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T08:32:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T08:32:12.151 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-09-11T08:32:12.171 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T08:32:12.181 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T08:32:12.186 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T08:32:12.187 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-09-11T08:32:12.189 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-09-11T08:32:12.206 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T08:32:12.212 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T08:32:12.212 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-09-11T08:32:12.247 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-09-11T08:32:12.281 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T08:32:12.337 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T08:32:12.350 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T08:32:12.351 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T08:32:12.351 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T08:32:12.351 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T08:32:12.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:32:12.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:32:12.369 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-09-11T08:32:12.369 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-11T08:32:12.427 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-503.el9.x86_64 2024-09-11T08:32:12.427 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.14.0-503.el9.x86_64 2024-09-11T08:32:12.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum install -y kernel 2024-09-11T08:32:12.428 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-503.el9.x86_64 2024-09-11T08:32:12.428 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-503.el9.x86_64 2024-09-11T08:32:12.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-09-11T08:32:14.681 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-11T08:32:16.954 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.2 MB 00:02 2024-09-11T08:32:23.623 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-11T08:32:25.737 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-11T08:32:29.973 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-11T08:32:30.162 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-11T08:32:31.416 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 19 kB 00:00 2024-09-11T08:32:31.597 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-11T08:32:32.303 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-11T08:32:32.700 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-11T08:32:37.726 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-11T08:32:38.302 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T08:32:39.757 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T08:32:39.758 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T08:32:39.855 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:32:39.866 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:32:39.866 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-11T08:32:39.866 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:32:39.866 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:32:39.866 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout:Install 4 Packages 2024-09-11T08:32:39.867 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:32:39.868 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 79 M 2024-09-11T08:32:39.868 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 126 M 2024-09-11T08:32:39.868 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:32:40.227 INFO:teuthology.orchestra.run.smithi086.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 276 kB/s | 23 kB 00:00 2024-09-11T08:32:40.379 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T08:32:40.379 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T08:32:40.471 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T08:32:40.475 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T08:32:40.476 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:32:40.476 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:32:40.476 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:32:40.476 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-11T08:32:40.476 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:32:40.476 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 79 M 2024-09-11T08:32:40.477 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 126 M 2024-09-11T08:32:40.477 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:32:40.585 INFO:teuthology.orchestra.run.smithi086.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-11T08:32:40.977 INFO:teuthology.orchestra.run.smithi086.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 41 MB/s | 35 MB 00:00 2024-09-11T08:32:41.291 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 43 kB/s | 23 kB 00:00 2024-09-11T08:32:41.403 INFO:teuthology.orchestra.run.smithi086.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-11T08:32:41.403 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:32:41.403 INFO:teuthology.orchestra.run.smithi086.stdout:Total 51 MB/s | 79 MB 00:01 2024-09-11T08:32:42.086 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:32:42.166 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T08:32:42.430 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:32:42.430 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:32:42.991 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 15 MB/s | 35 MB 00:02 2024-09-11T08:32:43.392 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:32:43.392 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:32:43.575 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-11T08:32:43.575 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:32:43.575 INFO:teuthology.orchestra.run.smithi045.stdout:Total 25 MB/s | 79 MB 00:03 2024-09-11T08:32:44.290 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:32:44.695 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:32:44.695 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:32:45.669 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:32:45.669 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:32:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:32:46.561 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:32:46.645 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:32:48.239 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:32:48.903 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:32:48.993 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:32:49.435 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:32:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:32:51.834 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:32:51.971 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:32:54.400 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:32:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:32:57.092 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:32:57.138 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:32:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:02.270 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:37.640 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:37.679 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:33:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:33:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:33:38.817 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:38.817 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:33:38.817 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:33:38.817 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T08:33:38.817 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T08:33:38.817 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T08:33:38.818 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T08:33:38.818 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:33:38.818 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:33:39.148 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:33:39.262 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:33:39.262 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:33:39.262 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T08:33:39.262 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T08:33:39.262 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T08:33:39.263 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T08:33:39.263 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T08:33:39.263 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:33:39.322 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:33:39.322 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:33:39.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-09-11T08:33:39.355 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-09-11T08:33:39.355 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-09-11T08:33:39.355 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.06 2024-09-11T08:33:39.355 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 82.el9 2024-09-11T08:33:39.355 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-09-11T08:33:39.355 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T08:33:39.355 INFO:teuthology.orchestra.run.smithi086.stdout:Group : Unspecified 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 8274670 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : builder@centos.org 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : CentOS 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:33:39.356 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T08:33:39.357 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T08:33:39.357 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T08:33:39.357 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-09-11T08:33:39.357 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:33:39.357 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T08:33:39.358 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T08:33:39.358 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 5.14.0-505.el9.x86_64 2024-09-11T08:33:39.358 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T08:33:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-09-11T08:33:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:41.272 INFO:teuthology.orchestra.run.smithi086.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T08:33:41.305 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-09-11T08:33:41.307 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-09-11T08:33:41.323 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.18l6CKJQDT 2024-09-11T08:33:41.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.18l6CKJQDT 2024-09-11T08:33:41.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.18l6CKJQDT 2024-09-11T08:33:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:33:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:33:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:33:41.536 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.18l6CKJQDT is 6KB 2024-09-11T08:33:41.585 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.18l6CKJQDT 2024-09-11T08:33:41.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T08:33:41.663 INFO:teuthology.orchestra.run.smithi086.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T08:33:41.663 INFO:teuthology.orchestra.run.smithi086.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T08:33:41.663 INFO:teuthology.orchestra.run.smithi086.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T08:33:41.663 INFO:teuthology.orchestra.run.smithi086.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T08:33:41.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T08:33:41.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-09-11T08:33:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:33:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:33:42.336 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:33:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T08:33:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T08:33:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T08:33:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T08:33:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:33:42.337 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:33:42.696 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:33:42.786 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:33:42.786 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:33:42.786 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T08:33:42.786 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T08:33:42.786 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T08:33:42.787 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T08:33:42.787 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T08:33:42.787 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:33:42.845 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:33:42.846 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:33:42.846 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 82.el9 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8274670 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T08:33:42.887 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T08:33:42.888 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T08:33:42.889 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-09-11T08:33:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:33:42.889 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T08:33:42.890 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T08:33:42.890 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-505.el9.x86_64 2024-09-11T08:33:42.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T08:33:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-09-11T08:33:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T08:33:44.892 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-09-11T08:33:44.893 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-11T08:33:44.908 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.buGHVHVRUk 2024-09-11T08:33:44.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.buGHVHVRUk 2024-09-11T08:33:44.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.buGHVHVRUk 2024-09-11T08:33:45.115 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.buGHVHVRUk is 6KB 2024-09-11T08:33:45.163 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.buGHVHVRUk 2024-09-11T08:33:45.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T08:33:45.240 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T08:33:45.240 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T08:33:45.240 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T08:33:45.240 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T08:33:45.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T08:33:45.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-09-11T08:34:11.851 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T08:34:11.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-11T08:34:11.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:34:15.411 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T08:34:15.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-11T08:34:15.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:34:30.337 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-09-11T08:34:33.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-09-11T08:34:39.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-11T08:34:39.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:34:42.397 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-09-11T08:34:42.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-11T08:34:42.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:34:54.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-11T08:34:54.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:35:26.215 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-09-11T08:35:26.970 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-11T08:35:26.970 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T08:35:26.970 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-09-11T08:35:27.017 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-505.el9.x86_64 2024-09-11T08:35:27.017 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T08:35:27.017 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T08:35:27.017 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T08:35:28.019 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T08:35:28.019 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T08:35:28.255 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2024-09-11T08:35:28.289 DEBUG:teuthology.parallel:result is None 2024-09-11T08:35:42.923 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T08:35:54.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-11T08:35:54.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:35:55.254 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-11T08:35:55.633 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-11T08:35:55.633 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T08:35:55.633 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-11T08:35:55.650 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-505.el9.x86_64 2024-09-11T08:35:55.650 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T08:35:55.650 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T08:35:55.650 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T08:35:56.651 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T08:35:56.783 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T08:35:56.858 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2024-09-11T08:35:56.882 DEBUG:teuthology.parallel:result is None 2024-09-11T08:35:56.882 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T08:35:56.890 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T08:35:56.891 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T08:35:56.893 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T08:35:56.912 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T08:35:56.918 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T08:35:56.927 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T08:35:56.928 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T08:35:56.953 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T08:35:56.982 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T08:35:56.988 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T08:35:56.988 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T08:35:57.018 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T08:35:57.053 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:35:57.054 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:35:57.065 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:35:57.066 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:35:57.068 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T08:35:57.075 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T08:35:57.075 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T08:35:57.110 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T08:35:57.138 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T08:35:57.147 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T08:35:57.147 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T08:35:57.180 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T08:35:57.196 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:35:57.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:35:57.325 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:35:57.408 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:35:57.450 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:35:57.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T08:35:57.517 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:35:57.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:35:57.618 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:35:57.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:35:57.744 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:35:57.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T08:35:57.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-09-11T08:35:57.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-09-11T08:35:57.856 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:35:57.908 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:35:58.322 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T08:35:58.330 INFO:teuthology.task.internal:Starting timer... 2024-09-11T08:35:58.330 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T08:35:58.348 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T08:35:58.356 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-11T08:35:58.356 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-09-11T08:35:58.407 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-09-11T08:35:58.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-09-11T08:35:58.461 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-09-11T08:35:58.463 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T08:35:58.463 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-09-11T08:35:58.483 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-09-11T08:35:58.484 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-09-11T08:35:58.514 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-09-11T08:35:58.514 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi086.front.sepia.ceph.com': 'permissive'} 2024-09-11T08:35:58.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T08:35:58.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:35:58.553 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T08:35:58.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:35:58.581 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T08:35:58.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-09-11T08:35:58.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-09-11T08:35:58.669 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T08:35:58.678 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T08:35:58.689 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T08:35:58.690 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi086.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T08:39:43.720 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com')] 2024-09-11T08:39:43.721 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-11T08:39:43.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:39:43.813 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-11T08:39:43.894 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-11T08:39:43.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-11T08:39:43.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:39:43.978 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-09-11T08:39:44.052 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-11T08:39:44.052 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T08:39:44.064 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T08:39:44.064 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:39:44.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:39:44.067 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:39:44.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:39:44.100 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:39:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:39:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:39:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:39:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-09-11T08:39:44.156 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-09-11T08:39:44.159 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-09-11T08:39:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-09-11T08:39:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:39:44.184 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:39:44.187 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:39:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:39:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:39:44.252 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:39:44.252 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-11T08:39:44.252 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:39:44.252 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:39:44.252 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:39:44.252 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:39:44.256 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:39:44.259 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:39:44.259 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-09-11T08:39:44.260 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:39:44.260 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:39:44.260 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:39:44.260 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:39:44.261 INFO:teuthology.run_tasks:Running task install... 2024-09-11T08:39:44.269 DEBUG:teuthology.task.install:project ceph 2024-09-11T08:39:44.269 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T08:39:44.269 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T08:39:44.269 INFO:teuthology.task.install:Using flavor: default 2024-09-11T08:39:44.275 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T08:39:44.275 INFO:teuthology.task.install:extra packages: [] 2024-09-11T08:39:44.275 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T08:39:44.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T08:39:44.278 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T08:39:44.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T08:39:44.422 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T08:39:44.423 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T08:39:44.426 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T08:39:44.427 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T08:39:44.534 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T08:39:44.534 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:39:44.534 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T08:39:44.547 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T08:39:44.547 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:39:44.547 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T08:39:44.562 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T08:39:44.562 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T08:39:44.573 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T08:39:44.573 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T08:39:44.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T08:39:44.639 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T08:39:44.707 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T08:39:44.714 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T08:39:44.773 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-09-11T08:39:44.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-09-11T08:39:44.778 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-09-11T08:39:44.780 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-09-11T08:39:45.010 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-09-11T08:39:45.016 INFO:teuthology.orchestra.run.smithi086.stdout:45 files removed 2024-09-11T08:39:45.038 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw 2024-09-11T08:39:45.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw 2024-09-11T08:39:45.594 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-11T08:39:45.598 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 319 kB/s | 84 kB 00:00 2024-09-11T08:39:45.873 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-11T08:39:45.886 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-11T08:39:46.162 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-11T08:39:46.199 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 6.6 kB/s | 1.9 kB 00:00 2024-09-11T08:39:46.779 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T08:39:48.830 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-11T08:39:49.557 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.2 MB 00:03 2024-09-11T08:39:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-11T08:39:55.257 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 19 kB 00:00 2024-09-11T08:39:56.061 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-11T08:40:01.501 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-11T08:40:03.886 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:40:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T08:40:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T08:40:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:Install 30 Packages 2024-09-11T08:40:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 2 Packages 2024-09-11T08:40:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:03.905 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 60 M 2024-09-11T08:40:03.905 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:40:04.544 INFO:teuthology.orchestra.run.smithi086.stdout:(1/32): ceph-base-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 5.4 MB 00:00 2024-09-11T08:40:04.602 INFO:teuthology.orchestra.run.smithi086.stdout:(2/32): ceph-selinux-19.1.1-298.gb53b6d3e.el9.x 433 kB/s | 25 kB 00:00 2024-09-11T08:40:04.702 INFO:teuthology.orchestra.run.smithi086.stdout:(3/32): libcephfs2-19.1.1-298.gb53b6d3e.el9.x86 7.4 MB/s | 752 kB 00:00 2024-09-11T08:40:04.786 INFO:teuthology.orchestra.run.smithi086.stdout:(4/32): libradosstriper1-19.1.1-298.gb53b6d3e.e 5.9 MB/s | 506 kB 00:00 2024-09-11T08:40:05.119 INFO:teuthology.orchestra.run.smithi086.stdout:(5/32): ceph-radosgw-19.1.1-298.gb53b6d3e.el9.x 10 MB/s | 11 MB 00:01 2024-09-11T08:40:05.336 INFO:teuthology.orchestra.run.smithi086.stdout:(6/32): ceph-common-19.1.1-298.gb53b6d3e.el9.x8 17 MB/s | 21 MB 00:01 2024-09-11T08:40:05.361 INFO:teuthology.orchestra.run.smithi086.stdout:(7/32): python3-ceph-argparse-19.1.1-298.gb53b6 187 kB/s | 45 kB 00:00 2024-09-11T08:40:05.387 INFO:teuthology.orchestra.run.smithi086.stdout:(8/32): python3-ceph-common-19.1.1-298.gb53b6d3 2.6 MB/s | 136 kB 00:00 2024-09-11T08:40:05.412 INFO:teuthology.orchestra.run.smithi086.stdout:(9/32): python3-cephfs-19.1.1-298.gb53b6d3e.el9 3.1 MB/s | 162 kB 00:00 2024-09-11T08:40:05.488 INFO:teuthology.orchestra.run.smithi086.stdout:(10/32): librgw2-19.1.1-298.gb53b6d3e.el9.x86_6 7.6 MB/s | 5.3 MB 00:00 2024-09-11T08:40:05.513 INFO:teuthology.orchestra.run.smithi086.stdout:(11/32): python3-rados-19.1.1-298.gb53b6d3e.el9 2.5 MB/s | 321 kB 00:00 2024-09-11T08:40:05.538 INFO:teuthology.orchestra.run.smithi086.stdout:(12/32): python3-rbd-19.1.1-298.gb53b6d3e.el9.x 2.3 MB/s | 301 kB 00:00 2024-09-11T08:40:05.563 INFO:teuthology.orchestra.run.smithi086.stdout:(13/32): python3-rgw-19.1.1-298.gb53b6d3e.el9.x 1.9 MB/s | 100 kB 00:00 2024-09-11T08:40:05.939 INFO:teuthology.orchestra.run.smithi086.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 121 kB/s | 45 kB 00:00 2024-09-11T08:40:05.972 INFO:teuthology.orchestra.run.smithi086.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 233 kB/s | 107 kB 00:00 2024-09-11T08:40:06.081 INFO:teuthology.orchestra.run.smithi086.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 295 kB/s | 160 kB 00:00 2024-09-11T08:40:06.206 INFO:teuthology.orchestra.run.smithi086.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 337 kB/s | 42 kB 00:00 2024-09-11T08:40:06.257 INFO:teuthology.orchestra.run.smithi086.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-09-11T08:40:06.399 INFO:teuthology.orchestra.run.smithi086.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 253 kB/s | 36 kB 00:00 2024-09-11T08:40:06.424 INFO:teuthology.orchestra.run.smithi086.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-09-11T08:40:06.449 INFO:teuthology.orchestra.run.smithi086.stdout:(21/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 662 kB 00:00 2024-09-11T08:40:06.499 INFO:teuthology.orchestra.run.smithi086.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 4.0 MB/s | 309 kB 00:00 2024-09-11T08:40:06.525 INFO:teuthology.orchestra.run.smithi086.stdout:(23/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 377 kB/s | 47 kB 00:00 2024-09-11T08:40:06.550 INFO:teuthology.orchestra.run.smithi086.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 534 kB/s | 26 kB 00:00 2024-09-11T08:40:06.584 INFO:teuthology.orchestra.run.smithi086.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T08:40:06.617 INFO:teuthology.orchestra.run.smithi086.stdout:(26/32): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-11T08:40:06.667 INFO:teuthology.orchestra.run.smithi086.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-09-11T08:40:06.693 INFO:teuthology.orchestra.run.smithi086.stdout:(28/32): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-11T08:40:06.760 INFO:teuthology.orchestra.run.smithi086.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-11T08:40:06.835 INFO:teuthology.orchestra.run.smithi086.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-11T08:40:06.943 INFO:teuthology.orchestra.run.smithi086.stdout:(31/32): librados2-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-11T08:40:07.136 INFO:teuthology.orchestra.run.smithi086.stdout:(32/32): librbd1-19.1.1-298.gb53b6d3e.el9.x86_6 8.4 MB/s | 3.2 MB 00:00 2024-09-11T08:40:07.141 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:40:07.141 INFO:teuthology.orchestra.run.smithi086.stdout:Total 19 MB/s | 60 MB 00:03 2024-09-11T08:40:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:40:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:40:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:40:07.959 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:40:07.960 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:40:09.419 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:40:09.503 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-11T08:40:09.541 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-11T08:40:09.574 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-11T08:40:09.605 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T08:40:09.640 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T08:40:09.835 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T08:40:09.888 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T08:40:10.250 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-11T08:40:10.369 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T08:40:10.455 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T08:40:10.550 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/34 2024-09-11T08:40:10.579 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T08:40:10.642 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T08:40:10.679 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-11T08:40:10.729 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 11/34 2024-09-11T08:40:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T08:40:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T08:40:10.887 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T08:40:10.929 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-11T08:40:11.038 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-11T08:40:11.073 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-11T08:40:11.114 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-11T08:40:11.146 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-09-11T08:40:11.175 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-11T08:40:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-11T08:40:11.339 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 20 MB 00:20 2024-09-11T08:40:11.543 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-11T08:40:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T08:40:11.802 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T08:40:11.834 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/34 2024-09-11T08:40:11.870 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-11T08:40:12.348 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-11T08:40:12.373 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T08:40:12.444 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T08:40:12.513 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/34 2024-09-11T08:40:12.543 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 28/34 2024-09-11T08:40:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T08:40:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T08:40:15.447 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T08:40:15.483 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T08:40:15.579 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T08:40:15.579 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T08:40:15.579 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T08:40:15.579 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:15.609 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T08:40:16.165 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.5 MB 00:00 2024-09-11T08:40:17.691 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-11T08:40:18.587 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-11T08:40:24.310 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-11T08:40:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T08:40:25.424 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-09-11T08:40:25.424 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-09-11T08:40:25.424 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-09-11T08:40:25.425 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-09-11T08:40:25.425 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-09-11T08:40:25.425 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-09-11T08:40:25.425 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-09-11T08:40:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:26.380 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T08:40:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T08:40:26.405 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:40:26.406 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T08:40:26.406 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:40:26.406 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:40:26.406 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:26.407 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T08:40:26.490 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T08:40:26.490 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T08:40:26.755 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T08:40:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T08:40:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T08:40:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout:Install 30 Packages 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 2 Packages 2024-09-11T08:40:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:26.763 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 60 M 2024-09-11T08:40:26.763 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:40:27.520 INFO:teuthology.orchestra.run.smithi045.stdout:(1/32): ceph-base-19.1.1-298.gb53b6d3e.el9.x86_ 9.1 MB/s | 5.4 MB 00:00 2024-09-11T08:40:27.570 INFO:teuthology.orchestra.run.smithi045.stdout:(2/32): ceph-selinux-19.1.1-298.gb53b6d3e.el9.x 504 kB/s | 25 kB 00:00 2024-09-11T08:40:27.670 INFO:teuthology.orchestra.run.smithi045.stdout:(3/32): libcephfs2-19.1.1-298.gb53b6d3e.el9.x86 7.4 MB/s | 752 kB 00:00 2024-09-11T08:40:27.762 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T08:40:27.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 8/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 9/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T08:40:27.763 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-11T08:40:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-11T08:40:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-11T08:40:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-11T08:40:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-11T08:40:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T08:40:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-11T08:40:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/34 2024-09-11T08:40:27.795 INFO:teuthology.orchestra.run.smithi045.stdout:(4/32): ceph-radosgw-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 11 MB 00:00 2024-09-11T08:40:27.837 INFO:teuthology.orchestra.run.smithi045.stdout:(5/32): libradosstriper1-19.1.1-298.gb53b6d3e.e 3.0 MB/s | 506 kB 00:00 2024-09-11T08:40:27.896 INFO:teuthology.orchestra.run.smithi045.stdout:(6/32): python3-ceph-argparse-19.1.1-298.gb53b6 774 kB/s | 45 kB 00:00 2024-09-11T08:40:28.096 INFO:teuthology.orchestra.run.smithi045.stdout:(7/32): ceph-common-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 21 MB 00:01 2024-09-11T08:40:28.121 INFO:teuthology.orchestra.run.smithi045.stdout:(8/32): python3-ceph-common-19.1.1-298.gb53b6d3 603 kB/s | 136 kB 00:00 2024-09-11T08:40:28.147 INFO:teuthology.orchestra.run.smithi045.stdout:(9/32): python3-cephfs-19.1.1-298.gb53b6d3e.el9 3.2 MB/s | 162 kB 00:00 2024-09-11T08:40:28.172 INFO:teuthology.orchestra.run.smithi045.stdout:(10/32): python3-rados-19.1.1-298.gb53b6d3e.el9 6.2 MB/s | 321 kB 00:00 2024-09-11T08:40:28.197 INFO:teuthology.orchestra.run.smithi045.stdout:(11/32): python3-rbd-19.1.1-298.gb53b6d3e.el9.x 5.8 MB/s | 301 kB 00:00 2024-09-11T08:40:28.214 INFO:teuthology.orchestra.run.smithi045.stdout:(12/32): python3-rgw-19.1.1-298.gb53b6d3e.el9.x 2.3 MB/s | 100 kB 00:00 2024-09-11T08:40:28.307 INFO:teuthology.orchestra.run.smithi045.stdout:(13/32): librgw2-19.1.1-298.gb53b6d3e.el9.x86_6 10 MB/s | 5.3 MB 00:00 2024-09-11T08:40:28.415 INFO:teuthology.orchestra.run.smithi045.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 492 kB/s | 107 kB 00:00 2024-09-11T08:40:28.440 INFO:teuthology.orchestra.run.smithi045.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-09-11T08:40:28.466 INFO:teuthology.orchestra.run.smithi045.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 638 kB/s | 160 kB 00:00 2024-09-11T08:40:28.516 INFO:teuthology.orchestra.run.smithi045.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 843 kB/s | 42 kB 00:00 2024-09-11T08:40:28.576 INFO:teuthology.orchestra.run.smithi045.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-11T08:40:28.625 INFO:teuthology.orchestra.run.smithi045.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-11T08:40:28.650 INFO:teuthology.orchestra.run.smithi045.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 476 kB/s | 36 kB 00:00 2024-09-11T08:40:28.675 INFO:teuthology.orchestra.run.smithi045.stdout:(21/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-09-11T08:40:28.701 INFO:teuthology.orchestra.run.smithi045.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-11T08:40:28.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T08:40:28.723 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:28.723 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T08:40:28.724 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.725 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:28.726 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:40:28.759 INFO:teuthology.orchestra.run.smithi045.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-09-11T08:40:28.785 INFO:teuthology.orchestra.run.smithi045.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-11T08:40:28.810 INFO:teuthology.orchestra.run.smithi045.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 278 kB/s | 26 kB 00:00 2024-09-11T08:40:28.827 INFO:teuthology.orchestra.run.smithi045.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-11T08:40:28.861 INFO:teuthology.orchestra.run.smithi045.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-11T08:40:28.895 INFO:teuthology.orchestra.run.smithi045.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-11T08:40:28.953 INFO:teuthology.orchestra.run.smithi045.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-11T08:40:29.036 INFO:teuthology.orchestra.run.smithi045.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-11T08:40:29.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-test 2024-09-11T08:40:29.103 INFO:teuthology.orchestra.run.smithi045.stdout:(31/32): librados2-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 3.4 MB 00:00 2024-09-11T08:40:29.195 INFO:teuthology.orchestra.run.smithi045.stdout:(32/32): librbd1-19.1.1-298.gb53b6d3e.el9.x86_6 13 MB/s | 3.2 MB 00:00 2024-09-11T08:40:29.200 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:40:29.200 INFO:teuthology.orchestra.run.smithi045.stdout:Total 25 MB/s | 60 MB 00:02 2024-09-11T08:40:29.306 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:40:29.345 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:40:29.345 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:40:29.643 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:40:29.774 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:40:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:Install 4 Packages 2024-09-11T08:40:29.776 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 49 M 2024-09-11T08:40:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 209 M 2024-09-11T08:40:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:40:30.034 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:40:30.035 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:40:30.088 INFO:teuthology.orchestra.run.smithi086.stdout:(1/4): libcephsqlite-19.1.1-298.gb53b6d3e.el9.x 770 kB/s | 163 kB 00:00 2024-09-11T08:40:30.113 INFO:teuthology.orchestra.run.smithi086.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-11T08:40:30.230 INFO:teuthology.orchestra.run.smithi086.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 449 kB/s | 64 kB 00:00 2024-09-11T08:40:31.364 INFO:teuthology.orchestra.run.smithi086.stdout:(4/4): ceph-test-19.1.1-298.gb53b6d3e.el9.x86_6 33 MB/s | 49 MB 00:01 2024-09-11T08:40:31.366 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:40:31.366 INFO:teuthology.orchestra.run.smithi086.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-11T08:40:31.395 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:40:31.415 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:40:31.415 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:40:31.550 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:40:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-11T08:40:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-11T08:40:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-11T08:40:31.707 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T08:40:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T08:40:31.797 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:40:31.797 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:40:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T08:40:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T08:40:32.338 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:40:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-11T08:40:32.408 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T08:40:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T08:40:32.455 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T08:40:32.482 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T08:40:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T08:40:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/34 2024-09-11T08:40:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T08:40:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T08:40:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-11T08:40:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 11/34 2024-09-11T08:40:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T08:40:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T08:40:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T08:40:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-11T08:40:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-11T08:40:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-11T08:40:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-11T08:40:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-09-11T08:40:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-11T08:40:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-11T08:40:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-11T08:40:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T08:40:33.730 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T08:40:33.760 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/34 2024-09-11T08:40:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-11T08:40:34.302 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-11T08:40:34.333 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T08:40:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T08:40:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/34 2024-09-11T08:40:34.527 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 28/34 2024-09-11T08:40:36.959 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T08:40:36.999 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T08:40:37.089 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T08:40:37.164 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T08:40:37.614 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T08:40:37.652 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T08:40:37.743 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T08:40:37.743 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T08:40:37.744 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T08:40:37.744 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:37.770 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T08:40:39.543 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T08:40:39.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/4 2024-09-11T08:40:39.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/4 2024-09-11T08:40:39.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:39.870 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:40:40.160 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph 2024-09-11T08:40:40.729 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:39 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:40:40.873 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:40:40.875 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:40.875 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:40:40.875 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:40.875 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T08:40:40.876 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T08:40:40.877 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T08:40:40.878 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T08:40:40.879 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:40.879 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:40:40.879 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:40.879 INFO:teuthology.orchestra.run.smithi086.stdout:Install 37 Packages 2024-09-11T08:40:40.879 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:40.879 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 30 M 2024-09-11T08:40:40.879 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 108 M 2024-09-11T08:40:40.880 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:40:41.473 INFO:teuthology.orchestra.run.smithi086.stdout:(1/37): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.rp 53 kB/s | 6.5 kB 00:00 2024-09-11T08:40:41.674 INFO:teuthology.orchestra.run.smithi086.stdout:(2/37): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-11T08:40:41.724 INFO:teuthology.orchestra.run.smithi086.stdout:(3/37): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_6 6.4 MB/s | 2.4 MB 00:00 2024-09-11T08:40:41.799 INFO:teuthology.orchestra.run.smithi086.stdout:(4/37): ceph-mgr-modules-core-19.1.1-298.gb53b6 3.2 MB/s | 249 kB 00:00 2024-09-11T08:40:41.891 INFO:teuthology.orchestra.run.smithi086.stdout:(5/37): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-11T08:40:42.050 INFO:teuthology.orchestra.run.smithi086.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-11T08:40:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:(7/37): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-11T08:40:42.259 INFO:teuthology.orchestra.run.smithi086.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 511 kB/s | 106 kB 00:00 2024-09-11T08:40:42.334 INFO:teuthology.orchestra.run.smithi086.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.6 MB/s | 125 kB 00:00 2024-09-11T08:40:42.393 INFO:teuthology.orchestra.run.smithi086.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 852 kB/s | 135 kB 00:00 2024-09-11T08:40:42.443 INFO:teuthology.orchestra.run.smithi086.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-09-11T08:40:42.477 INFO:teuthology.orchestra.run.smithi086.stdout:(12/37): python3-cryptography-36.0.1-4.el9.x86_ 2.1 MB/s | 1.2 MB 00:00 2024-09-11T08:40:42.593 INFO:teuthology.orchestra.run.smithi086.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 298 kB/s | 35 kB 00:00 2024-09-11T08:40:42.619 INFO:teuthology.orchestra.run.smithi086.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 836 kB/s | 188 kB 00:00 2024-09-11T08:40:42.644 INFO:teuthology.orchestra.run.smithi086.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 858 kB/s | 172 kB 00:00 2024-09-11T08:40:42.677 INFO:teuthology.orchestra.run.smithi086.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 919 kB/s | 77 kB 00:00 2024-09-11T08:40:42.703 INFO:teuthology.orchestra.run.smithi086.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-11T08:40:42.736 INFO:teuthology.orchestra.run.smithi086.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 434 kB/s | 51 kB 00:00 2024-09-11T08:40:42.761 INFO:teuthology.orchestra.run.smithi086.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-11T08:40:42.804 INFO:teuthology.orchestra.run.smithi086.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-11T08:40:42.829 INFO:teuthology.orchestra.run.smithi086.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 669 kB/s | 43 kB 00:00 2024-09-11T08:40:42.854 INFO:teuthology.orchestra.run.smithi086.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 433 kB/s | 11 kB 00:00 2024-09-11T08:40:42.879 INFO:teuthology.orchestra.run.smithi086.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 706 kB/s | 18 kB 00:00 2024-09-11T08:40:42.905 INFO:teuthology.orchestra.run.smithi086.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-11T08:40:42.930 INFO:teuthology.orchestra.run.smithi086.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-11T08:40:42.955 INFO:teuthology.orchestra.run.smithi086.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-11T08:40:42.980 INFO:teuthology.orchestra.run.smithi086.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-11T08:40:43.006 INFO:teuthology.orchestra.run.smithi086.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-09-11T08:40:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-11T08:40:43.056 INFO:teuthology.orchestra.run.smithi086.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-11T08:40:43.081 INFO:teuthology.orchestra.run.smithi086.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-11T08:40:43.107 INFO:teuthology.orchestra.run.smithi086.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-11T08:40:43.132 INFO:teuthology.orchestra.run.smithi086.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-11T08:40:43.157 INFO:teuthology.orchestra.run.smithi086.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-11T08:40:43.183 INFO:teuthology.orchestra.run.smithi086.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T08:40:43.208 INFO:teuthology.orchestra.run.smithi086.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 396 kB/s | 20 kB 00:00 2024-09-11T08:40:43.233 INFO:teuthology.orchestra.run.smithi086.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-11T08:40:43.237 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:40:43.237 INFO:teuthology.orchestra.run.smithi086.stdout:Total 13 MB/s | 30 MB 00:02 2024-09-11T08:40:43.387 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:40:43.427 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:40:43.427 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:40:43.933 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:40:43.933 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:40:45.293 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:40:45.369 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-11T08:40:45.403 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-11T08:40:45.433 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-11T08:40:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-11T08:40:45.540 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-11T08:40:45.573 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-11T08:40:45.665 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-11T08:40:45.744 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-11T08:40:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-11T08:40:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-11T08:40:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-11T08:40:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-11T08:40:45.925 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-11T08:40:45.974 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-11T08:40:46.017 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-11T08:40:46.052 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-11T08:40:46.100 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-11T08:40:46.141 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-11T08:40:46.213 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-11T08:40:46.351 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-11T08:40:46.431 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-11T08:40:46.485 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-11T08:40:46.523 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-11T08:40:46.572 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-11T08:40:46.636 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-11T08:40:46.821 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-11T08:40:46.876 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-11T08:40:46.946 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-11T08:40:47.207 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-11T08:40:47.364 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 32/37 2024-09-11T08:40:47.393 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T08:40:47.420 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T08:40:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:40:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T08:40:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:40:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:40:47.421 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:48.381 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T08:40:48.381 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-09-11T08:40:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-09-11T08:40:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-09-11T08:40:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-09-11T08:40:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-09-11T08:40:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-09-11T08:40:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-09-11T08:40:48.382 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:48.862 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T08:40:48.889 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T08:40:48.889 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:40:48.889 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T08:40:48.889 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:40:48.889 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:40:48.889 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:49.328 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T08:40:49.352 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T08:40:49.352 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:40:49.352 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T08:40:49.353 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:40:49.353 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:40:49.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T08:40:49.413 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T08:40:49.414 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:40:49.414 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T08:40:49.414 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:40:49.414 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:40:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T08:40:49.506 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T08:40:49.506 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T08:40:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T08:40:49.611 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T08:40:49.612 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:40:49.612 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T08:40:49.612 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:40:49.612 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:40:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:49.676 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T08:40:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T08:40:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/34 2024-09-11T08:40:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/34 2024-09-11T08:40:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/34 2024-09-11T08:40:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/34 2024-09-11T08:40:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/34 2024-09-11T08:40:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 8/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 9/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-11T08:40:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-11T08:40:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-11T08:40:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-11T08:40:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-11T08:40:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-09-11T08:40:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-11T08:40:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-11T08:40:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-11T08:40:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-11T08:40:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-11T08:40:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-11T08:40:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-11T08:40:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T08:40:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-11T08:40:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/34 2024-09-11T08:40:51.403 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T08:40:51.403 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/37 2024-09-11T08:40:51.403 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/37 2024-09-11T08:40:51.403 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/37 2024-09-11T08:40:51.403 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/37 2024-09-11T08:40:51.403 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/37 2024-09-11T08:40:51.403 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 6/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-11T08:40:51.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-11T08:40:51.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-11T08:40:51.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-11T08:40:51.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-11T08:40:51.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-11T08:40:51.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-11T08:40:51.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-11T08:40:51.406 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-11T08:40:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-11T08:40:51.408 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-11T08:40:51.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T08:40:51.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:51.767 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-09-11T08:40:51.767 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.767 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.767 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T08:40:51.768 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.769 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T08:40:51.770 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:51.771 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:40:52.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-test 2024-09-11T08:40:52.531 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-11T08:40:52.531 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:52.531 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:40:52.531 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:52.531 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:52.531 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T08:40:52.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T08:40:52.533 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:52.534 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:40:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:40:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:40:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:40:52.864 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:40:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:40:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:40:52.864 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T08:40:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:40:52.868 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T08:40:52.868 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T08:40:52.868 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T08:40:52.869 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:40:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:40:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-11T08:40:52.869 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:40:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 49 M 2024-09-11T08:40:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 209 M 2024-09-11T08:40:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:40:52.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-base 2024-09-11T08:40:53.020 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-11T08:40:53.078 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-11T08:40:53.103 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): libcephsqlite-19.1.1-298.gb53b6d3e.el9.x 755 kB/s | 163 kB 00:00 2024-09-11T08:40:53.495 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:40:53.584 INFO:teuthology.orchestra.run.smithi086.stdout:Package ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:40:53.628 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:40:53.629 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:40:53.629 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:40:53.704 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install cephadm 2024-09-11T08:40:54.260 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:40:54.393 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:40:54.393 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:54.393 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:40:54.393 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 765 k 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 771 k 2024-09-11T08:40:54.394 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:40:54.425 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): ceph-test-19.1.1-298.gb53b6d3e.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-11T08:40:54.427 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:40:54.428 INFO:teuthology.orchestra.run.smithi045.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-11T08:40:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:40:54.478 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:40:54.478 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:40:54.661 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm-19.1.1-298.gb53b6d3e.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-11T08:40:54.661 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:40:54.662 INFO:teuthology.orchestra.run.smithi086.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-11T08:40:54.662 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:40:54.667 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:40:54.667 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:40:54.673 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:40:54.673 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:40:54.743 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:40:54.873 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:40:54.874 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:40:55.202 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:40:55.289 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:40:55.435 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:40:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T08:40:55.562 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T08:40:55.589 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T08:40:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:40:55.936 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:40:55.936 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:55.936 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:40:55.936 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:40:55.936 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:55.936 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:40:56.197 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T08:40:56.765 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:40:56.898 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:40:56.898 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repo Size 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 145 k 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 435 k 2024-09-11T08:40:56.899 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:40:57.108 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-immutable-object-cache-19.1.1-298.gb53b6d3 696 kB/s | 145 kB 00:00 2024-09-11T08:40:57.108 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:40:57.109 INFO:teuthology.orchestra.run.smithi086.stdout:Total 691 kB/s | 145 kB 00:00 2024-09-11T08:40:57.109 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:40:57.116 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:40:57.117 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:40:57.173 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:40:57.173 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:40:57.356 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:40:57.467 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T08:40:57.493 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T08:40:57.493 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:40:57.493 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T08:40:57.493 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:58.402 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T08:40:58.402 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:58.402 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:40:58.402 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:40:58.402 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:40:58.403 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:40:58.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr 2024-09-11T08:40:59.203 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:58 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:40:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:Package ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:40:59.336 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:40:59.337 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:40:59.337 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:40:59.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T08:40:59.960 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:58 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:00.092 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:00.093 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:41:00.093 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:00.093 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T08:41:00.093 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:41:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:00.094 INFO:teuthology.orchestra.run.smithi086.stdout:Install 12 Packages 2024-09-11T08:41:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:00.095 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 8.7 M 2024-09-11T08:41:00.095 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 104 M 2024-09-11T08:41:00.095 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:00.497 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T08:41:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T08:41:00.661 INFO:teuthology.orchestra.run.smithi086.stdout:(1/12): ceph-prometheus-alerts-19.1.1-298.gb53b 84 kB/s | 16 kB 00:00 2024-09-11T08:41:00.686 INFO:teuthology.orchestra.run.smithi086.stdout:(2/12): ceph-grafana-dashboards-19.1.1-298.gb53 144 kB/s | 32 kB 00:00 2024-09-11T08:41:00.861 INFO:teuthology.orchestra.run.smithi086.stdout:(3/12): ceph-mgr-dashboard-19.1.1-298.gb53b6d3e 9.0 MB/s | 3.6 MB 00:00 2024-09-11T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.8 MB/s | 863 kB 00:00 2024-09-11T08:41:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-11T08:41:01.237 INFO:teuthology.orchestra.run.smithi086.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-11T08:41:01.304 INFO:teuthology.orchestra.run.smithi086.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-11T08:41:01.329 INFO:teuthology.orchestra.run.smithi086.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-11T08:41:01.346 INFO:teuthology.orchestra.run.smithi086.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.8 MB/s | 31 kB 00:00 2024-09-11T08:41:01.380 INFO:teuthology.orchestra.run.smithi086.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.6 MB/s | 188 kB 00:00 2024-09-11T08:41:01.488 INFO:teuthology.orchestra.run.smithi086.stdout:(11/12): python3-protobuf-3.14.0-13.el9.noarch. 336 kB/s | 269 kB 00:00 2024-09-11T08:41:01.647 INFO:teuthology.orchestra.run.smithi086.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-09-11T08:41:01.651 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:01.651 INFO:teuthology.orchestra.run.smithi086.stdout:Total 5.6 MB/s | 8.7 MB 00:01 2024-09-11T08:41:01.700 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:01.717 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:01.717 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:01.918 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:01.918 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:02.539 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-11T08:41:02.582 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-11T08:41:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-11T08:41:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-11T08:41:02.774 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-11T08:41:02.906 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-11T08:41:02.948 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T08:41:02.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/4 2024-09-11T08:41:02.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/4 2024-09-11T08:41:02.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T08:41:02.999 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-11T08:41:03.041 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-11T08:41:03.069 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-11T08:41:03.109 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 10/12 2024-09-11T08:41:03.252 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T08:41:03.252 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:03.252 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:03.252 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:03.252 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:03.253 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T08:41:03.253 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T08:41:03.253 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:03.253 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:03.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph 2024-09-11T08:41:04.159 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:40 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 11/12 2024-09-11T08:41:04.210 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T08:41:04.303 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:04.305 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:04.305 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:41:04.305 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:04.305 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T08:41:04.306 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T08:41:04.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T08:41:04.308 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout:Install 37 Packages 2024-09-11T08:41:04.309 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:04.310 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 30 M 2024-09-11T08:41:04.310 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 108 M 2024-09-11T08:41:04.310 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:05.014 INFO:teuthology.orchestra.run.smithi045.stdout:(1/37): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-11T08:41:05.223 INFO:teuthology.orchestra.run.smithi045.stdout:(2/37): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_6 4.6 MB/s | 1.5 MB 00:00 2024-09-11T08:41:05.282 INFO:teuthology.orchestra.run.smithi045.stdout:(3/37): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_6 6.1 MB/s | 2.4 MB 00:00 2024-09-11T08:41:05.409 INFO:teuthology.orchestra.run.smithi045.stdout:(4/37): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-11T08:41:05.432 INFO:teuthology.orchestra.run.smithi045.stdout:(5/37): ceph-mgr-modules-core-19.1.1-298.gb53b6 1.6 MB/s | 249 kB 00:00 2024-09-11T08:41:05.574 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T08:41:05.574 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 1/12 2024-09-11T08:41:05.574 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 2/12 2024-09-11T08:41:05.574 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 3/12 2024-09-11T08:41:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-11T08:41:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-11T08:41:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-11T08:41:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-11T08:41:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-11T08:41:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-11T08:41:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-11T08:41:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-11T08:41:05.608 INFO:teuthology.orchestra.run.smithi045.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-11T08:41:05.791 INFO:teuthology.orchestra.run.smithi045.stdout:(7/37): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-11T08:41:05.817 INFO:teuthology.orchestra.run.smithi045.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 511 kB/s | 106 kB 00:00 2024-09-11T08:41:05.885 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-11T08:41:05.886 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:05.886 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:05.886 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T08:41:05.886 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:05.886 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:05.887 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:05.887 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T08:41:05.887 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T08:41:05.887 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T08:41:05.887 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T08:41:05.887 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T08:41:05.888 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T08:41:05.888 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T08:41:05.888 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T08:41:05.888 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:05.888 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.5 MB/s | 125 kB 00:00 2024-09-11T08:41:05.959 INFO:teuthology.orchestra.run.smithi045.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 810 kB/s | 135 kB 00:00 2024-09-11T08:41:06.018 INFO:teuthology.orchestra.run.smithi045.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 2.1 MB/s | 1.2 MB 00:00 2024-09-11T08:41:06.043 INFO:teuthology.orchestra.run.smithi045.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.5 MB/s | 215 kB 00:00 2024-09-11T08:41:06.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T08:41:06.160 INFO:teuthology.orchestra.run.smithi045.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 298 kB/s | 35 kB 00:00 2024-09-11T08:41:06.194 INFO:teuthology.orchestra.run.smithi045.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 804 kB/s | 188 kB 00:00 2024-09-11T08:41:06.218 INFO:teuthology.orchestra.run.smithi045.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 858 kB/s | 172 kB 00:00 2024-09-11T08:41:06.244 INFO:teuthology.orchestra.run.smithi045.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 921 kB/s | 77 kB 00:00 2024-09-11T08:41:06.270 INFO:teuthology.orchestra.run.smithi045.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-11T08:41:06.294 INFO:teuthology.orchestra.run.smithi045.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 550 kB/s | 42 kB 00:00 2024-09-11T08:41:06.345 INFO:teuthology.orchestra.run.smithi045.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-11T08:41:06.370 INFO:teuthology.orchestra.run.smithi045.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-11T08:41:06.404 INFO:teuthology.orchestra.run.smithi045.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 399 kB/s | 43 kB 00:00 2024-09-11T08:41:06.429 INFO:teuthology.orchestra.run.smithi045.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 470 kB/s | 11 kB 00:00 2024-09-11T08:41:06.454 INFO:teuthology.orchestra.run.smithi045.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 705 kB/s | 18 kB 00:00 2024-09-11T08:41:06.480 INFO:teuthology.orchestra.run.smithi045.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T08:41:06.505 INFO:teuthology.orchestra.run.smithi045.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-11T08:41:06.530 INFO:teuthology.orchestra.run.smithi045.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-11T08:41:06.555 INFO:teuthology.orchestra.run.smithi045.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-11T08:41:06.581 INFO:teuthology.orchestra.run.smithi045.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 916 kB/s | 46 kB 00:00 2024-09-11T08:41:06.606 INFO:teuthology.orchestra.run.smithi045.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-11T08:41:06.632 INFO:teuthology.orchestra.run.smithi045.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 664 kB/s | 16 kB 00:00 2024-09-11T08:41:06.657 INFO:teuthology.orchestra.run.smithi045.stdout:(31/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-09-11T08:41:06.682 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:05 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:06.682 INFO:teuthology.orchestra.run.smithi045.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-11T08:41:06.707 INFO:teuthology.orchestra.run.smithi045.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-11T08:41:06.733 INFO:teuthology.orchestra.run.smithi045.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-11T08:41:06.758 INFO:teuthology.orchestra.run.smithi045.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-11T08:41:06.783 INFO:teuthology.orchestra.run.smithi045.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T08:41:06.808 INFO:teuthology.orchestra.run.smithi045.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-11T08:41:06.812 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:06.812 INFO:teuthology.orchestra.run.smithi045.stdout:Total 12 MB/s | 30 MB 00:02 2024-09-11T08:41:06.817 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:06.818 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================== 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================== 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T08:41:06.819 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================== 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout:Install 12 Packages 2024-09-11T08:41:06.820 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:06.821 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 38 M 2024-09-11T08:41:06.821 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 212 M 2024-09-11T08:41:06.821 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:06.971 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:07.015 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:07.015 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 885 kB/s | 191 kB 00:00 2024-09-11T08:41:07.290 INFO:teuthology.orchestra.run.smithi086.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-11T08:41:07.315 INFO:teuthology.orchestra.run.smithi086.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-11T08:41:07.357 INFO:teuthology.orchestra.run.smithi086.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-09-11T08:41:07.448 INFO:teuthology.orchestra.run.smithi086.stdout:(5/12): ceph-mgr-diskprediction-local-19.1.1-29 16 MB/s | 7.4 MB 00:00 2024-09-11T08:41:07.473 INFO:teuthology.orchestra.run.smithi086.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 330 kB/s | 38 kB 00:00 2024-09-11T08:41:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:07.574 INFO:teuthology.orchestra.run.smithi086.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-11T08:41:07.774 INFO:teuthology.orchestra.run.smithi086.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.2 MB/s | 3.0 MB 00:00 2024-09-11T08:41:07.833 INFO:teuthology.orchestra.run.smithi086.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-09-11T08:41:07.966 INFO:teuthology.orchestra.run.smithi086.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-11T08:41:08.041 INFO:teuthology.orchestra.run.smithi086.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 8.3 MB/s | 4.9 MB 00:00 2024-09-11T08:41:08.600 INFO:teuthology.orchestra.run.smithi086.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-11T08:41:08.601 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:08.601 INFO:teuthology.orchestra.run.smithi086.stdout:Total 21 MB/s | 38 MB 00:01 2024-09-11T08:41:08.919 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:08.959 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:09.001 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-11T08:41:09.001 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:09.002 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:09.012 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-11T08:41:09.039 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-11T08:41:09.114 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-11T08:41:09.156 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-11T08:41:09.188 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-11T08:41:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-11T08:41:09.379 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-11T08:41:09.405 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:09.405 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:09.425 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-11T08:41:09.460 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-11T08:41:09.489 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-11T08:41:09.531 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-11T08:41:09.565 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-11T08:41:09.607 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-11T08:41:09.649 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-11T08:41:09.687 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-11T08:41:09.737 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-11T08:41:09.777 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-11T08:41:09.852 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-11T08:41:09.996 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-11T08:41:10.087 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-11T08:41:10.147 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-11T08:41:10.164 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:10.185 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-11T08:41:10.231 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:41:10.244 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-11T08:41:10.273 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:41:10.308 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-11T08:41:10.373 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:41:10.408 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:41:10.504 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-11T08:41:10.556 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-11T08:41:10.593 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:41:10.615 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:41:10.637 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-11T08:41:10.700 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:41:10.727 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-11T08:41:10.763 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-11T08:41:10.885 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-11T08:41:11.059 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 32/37 2024-09-11T08:41:11.090 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T08:41:11.117 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T08:41:11.117 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:41:11.117 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T08:41:11.117 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:41:11.117 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:41:11.117 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:41:11.477 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:41:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T08:41:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T08:41:12.670 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:41:12.670 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T08:41:12.670 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:41:12.670 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:41:12.670 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:12.707 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:41:13.125 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T08:41:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T08:41:13.151 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:41:13.151 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T08:41:13.151 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:41:13.151 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:41:13.151 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:13.392 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T08:41:13.417 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T08:41:13.417 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:41:13.417 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T08:41:13.417 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:41:13.418 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:41:13.418 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:13.492 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T08:41:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:41:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T08:41:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T08:41:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 6/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-11T08:41:15.256 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-11T08:41:15.257 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-11T08:41:15.259 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-11T08:41:16.182 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-11T08:41:16.182 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:16.182 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:16.182 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:16.182 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:16.182 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T08:41:16.183 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T08:41:16.184 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T08:41:16.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:16.186 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 1/12 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:41:16.404 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:41:16.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:41:16.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:41:16.405 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:41:16.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-base 2024-09-11T08:41:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:41:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:16.787 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:41:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:41:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:17.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:41:17.223 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:17.317 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:17.361 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:17.362 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:41:17.362 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:17.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install cephadm 2024-09-11T08:41:17.729 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:16 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:17.865 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:17.866 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:17.866 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:41:17.867 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout:Install 13 Packages 2024-09-11T08:41:17.868 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:17.869 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 2.2 M 2024-09-11T08:41:17.869 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 26 M 2024-09-11T08:41:17.869 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:18.025 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:18.161 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:18.162 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-11T08:41:18.163 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:18.163 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 765 k 2024-09-11T08:41:18.163 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 771 k 2024-09-11T08:41:18.163 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:18.413 INFO:teuthology.orchestra.run.smithi086.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 144 kB/s | 19 kB 00:00 2024-09-11T08:41:18.432 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm-19.1.1-298.gb53b6d3e.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-11T08:41:18.432 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:18.433 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-11T08:41:18.433 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:18.438 INFO:teuthology.orchestra.run.smithi086.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 161 kB/s | 26 kB 00:00 2024-09-11T08:41:18.438 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:18.439 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:18.445 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:18.445 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:18.464 INFO:teuthology.orchestra.run.smithi086.stdout:(3/13): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el9. 268 kB/s | 49 kB 00:00 2024-09-11T08:41:18.519 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:18.530 INFO:teuthology.orchestra.run.smithi086.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-11T08:41:18.564 INFO:teuthology.orchestra.run.smithi086.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-11T08:41:18.598 INFO:teuthology.orchestra.run.smithi086.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 801 kB/s | 54 kB 00:00 2024-09-11T08:41:18.648 INFO:teuthology.orchestra.run.smithi086.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 385 kB/s | 32 kB 00:00 2024-09-11T08:41:18.674 INFO:teuthology.orchestra.run.smithi086.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-09-11T08:41:18.699 INFO:teuthology.orchestra.run.smithi086.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-11T08:41:18.732 INFO:teuthology.orchestra.run.smithi086.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-11T08:41:18.757 INFO:teuthology.orchestra.run.smithi086.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-11T08:41:18.782 INFO:teuthology.orchestra.run.smithi086.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-11T08:41:18.824 INFO:teuthology.orchestra.run.smithi086.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-11T08:41:18.827 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:18.827 INFO:teuthology.orchestra.run.smithi086.stdout:Total 2.3 MB/s | 2.2 MB 00:00 2024-09-11T08:41:18.891 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:18.906 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:18.906 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:18.933 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:18.996 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:18.996 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:19.031 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:19.403 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:19.521 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:19.642 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:19.642 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:19.696 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:41:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:41:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:41:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:41:19.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T08:41:19.901 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:41:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:41:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:41:20.169 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:41:20.445 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:20.582 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:20.583 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-11T08:41:20.584 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:20.584 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 145 k 2024-09-11T08:41:20.584 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 435 k 2024-09-11T08:41:20.584 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:20.798 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-immutable-object-cache-19.1.1-298.gb53b6d3 680 kB/s | 145 kB 00:00 2024-09-11T08:41:20.799 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:20.799 INFO:teuthology.orchestra.run.smithi045.stdout:Total 674 kB/s | 145 kB 00:00 2024-09-11T08:41:20.799 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:20.807 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:20.807 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:20.868 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:20.868 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:20.889 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:41:20.950 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:41:20.992 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:41:21.028 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:21.049 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:41:21.078 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T08:41:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T08:41:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T08:41:21.126 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:41:21.126 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T08:41:21.126 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 1/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:41:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:41:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T08:41:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:22.102 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:22.102 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:22.102 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:41:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:22.189 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:22.190 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:22.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr 2024-09-11T08:41:22.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:41:22.947 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:58 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:23.025 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:22 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:23.041 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:23.085 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:23.086 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:41:23.086 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:23.152 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T08:41:23.160 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:23.160 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:23.160 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:41:23.160 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout:Install 6 Packages 2024-09-11T08:41:23.161 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:23.162 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 7.0 M 2024-09-11T08:41:23.162 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 33 M 2024-09-11T08:41:23.162 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:23.609 INFO:teuthology.orchestra.run.smithi086.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-11T08:41:23.651 INFO:teuthology.orchestra.run.smithi086.stdout:(2/6): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e.el 673 kB/s | 146 kB 00:00 2024-09-11T08:41:23.710 INFO:teuthology.orchestra.run.smithi086.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.3 MB/s | 548 kB 00:00 2024-09-11T08:41:23.735 INFO:teuthology.orchestra.run.smithi086.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 690 kB/s | 58 kB 00:00 2024-09-11T08:41:23.735 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:59 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:23.760 INFO:teuthology.orchestra.run.smithi086.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-11T08:41:23.860 INFO:teuthology.orchestra.run.smithi086.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-11T08:41:23.863 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:23.863 INFO:teuthology.orchestra.run.smithi086.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-11T08:41:23.874 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T08:41:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-09-11T08:41:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:23.878 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 8.7 M 2024-09-11T08:41:23.878 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 104 M 2024-09-11T08:41:23.878 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:23.935 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:23.946 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:23.946 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:24.047 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:24.048 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:24.270 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): ceph-grafana-dashboards-19.1.1-298.gb53 201 kB/s | 32 kB 00:00 2024-09-11T08:41:24.295 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): ceph-prometheus-alerts-19.1.1-298.gb53b 90 kB/s | 16 kB 00:00 2024-09-11T08:41:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:24.504 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): ceph-mgr-dashboard-19.1.1-298.gb53b6d3e 9.1 MB/s | 3.6 MB 00:00 2024-09-11T08:41:24.529 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.1 MB/s | 269 kB 00:00 2024-09-11T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:41:24.578 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:41:24.579 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-11T08:41:24.604 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-11T08:41:24.672 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-11T08:41:24.697 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2024-09-11T08:41:24.722 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-11T08:41:24.756 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.1 MB/s | 1.0 MB 00:00 2024-09-11T08:41:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-11T08:41:25.091 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:41:25.180 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:41:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:41:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T08:41:25.565 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 814 kB/s | 863 kB 00:01 2024-09-11T08:41:25.569 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:25.569 INFO:teuthology.orchestra.run.smithi045.stdout:Total 5.1 MB/s | 8.7 MB 00:01 2024-09-11T08:41:25.622 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:25.642 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:25.642 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:25.867 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:25.867 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:26.042 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T08:41:26.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/6 2024-09-11T08:41:26.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:41:26.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:41:26.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:41:26.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:41:26.305 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:41:26.305 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:26.305 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:26.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:26.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:41:26.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:41:26.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:41:26.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:41:26.306 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:41:26.306 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:26.306 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-11T08:41:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-11T08:41:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-11T08:41:26.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-fuse 2024-09-11T08:41:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-11T08:41:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-11T08:41:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-11T08:41:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-11T08:41:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-11T08:41:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-11T08:41:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 10/12 2024-09-11T08:41:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:26 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:27.263 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:27.264 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:41:27.264 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:27.264 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T08:41:27.264 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-11T08:41:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:41:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:27.265 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:27.265 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:27.265 INFO:teuthology.orchestra.run.smithi086.stdout:Install 2 Packages 2024-09-11T08:41:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:27.265 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 977 k 2024-09-11T08:41:27.265 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 2.9 M 2024-09-11T08:41:27.265 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:27.403 INFO:teuthology.orchestra.run.smithi086.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 676 kB/s | 80 kB 00:00 2024-09-11T08:41:28.137 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 11/12 2024-09-11T08:41:28.169 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 1/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 2/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 3/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-11T08:41:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-11T08:41:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T08:41:29.726 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T08:41:29.727 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T08:41:29.727 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T08:41:29.727 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T08:41:29.727 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T08:41:29.727 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:29.727 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:29.745 INFO:teuthology.orchestra.run.smithi086.stdout:(2/2): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86_6 365 kB/s | 897 kB 00:02 2024-09-11T08:41:29.746 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:29.746 INFO:teuthology.orchestra.run.smithi086.stdout:Total 394 kB/s | 977 kB 00:02 2024-09-11T08:41:29.761 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:29.786 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:29.786 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:29.872 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:29.872 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:29.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T08:41:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:41:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T08:41:30.556 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:06 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:30.697 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:30.698 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-11T08:41:30.698 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:41:30.698 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-11T08:41:30.698 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T08:41:30.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T08:41:30.700 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:30.700 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:30.700 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-11T08:41:30.700 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-09-11T08:41:30.700 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:30.700 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 38 M 2024-09-11T08:41:30.700 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 212 M 2024-09-11T08:41:30.701 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T08:41:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/2 2024-09-11T08:41:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:41:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:31.232 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:41:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:31.233 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:31.557 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-29 11 MB/s | 7.4 MB 00:00 2024-09-11T08:41:31.558 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-volume 2024-09-11T08:41:31.649 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 358 kB/s | 33 kB 00:00 2024-09-11T08:41:31.674 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 236 kB/s | 191 kB 00:00 2024-09-11T08:41:31.767 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 198 kB/s | 18 kB 00:00 2024-09-11T08:41:31.834 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 580 kB/s | 38 kB 00:00 2024-09-11T08:41:31.859 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 807 kB/s | 801 kB 00:00 2024-09-11T08:41:31.918 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-09-11T08:41:32.043 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-09-11T08:41:32.093 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-09-11T08:41:32.138 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:32.168 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 15 MB/s | 4.9 MB 00:00 2024-09-11T08:41:32.194 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-devel-3.9.19-8.el9.x86_64.rpm 732 kB/s | 245 kB 00:00 2024-09-11T08:41:32.278 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:32.279 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-11T08:41:32.279 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:32.279 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 285 k 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 1.3 M 2024-09-11T08:41:32.280 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:32.461 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-09-11T08:41:32.462 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:32.462 INFO:teuthology.orchestra.run.smithi045.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-11T08:41:32.532 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-volume-19.1.1-298.gb53b6d3e.el9.noarch.rpm 1.1 MB/s | 285 kB 00:00 2024-09-11T08:41:32.533 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:32.533 INFO:teuthology.orchestra.run.smithi086.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-11T08:41:32.534 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:32.541 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:32.542 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:32.573 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:32.574 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:32.794 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:32.836 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:32.864 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:32.864 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:32.909 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:32.909 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:41:32.909 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:41:32.909 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:33.314 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:33.314 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:33.931 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:33.931 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:33.931 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:33.931 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:33.932 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:33.932 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:34.107 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:34.182 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:41:34.226 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:41:34.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install librados-devel 2024-09-11T08:41:34.331 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:41:34.369 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:41:34.563 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:41:34.601 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:41:34.680 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:41:34.818 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repo Size 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-11T08:41:34.958 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 127 k 2024-09-11T08:41:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 456 k 2024-09-11T08:41:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:35.162 INFO:teuthology.orchestra.run.smithi086.stdout:librados-devel-19.1.1-298.gb53b6d3e.el9.x86_64. 621 kB/s | 127 kB 00:00 2024-09-11T08:41:35.162 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:35.162 INFO:teuthology.orchestra.run.smithi086.stdout:Total 617 kB/s | 127 kB 00:00 2024-09-11T08:41:35.163 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:35.168 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:35.168 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:35.232 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:35.232 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:35.379 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:41:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:35.503 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:41:35.510 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:36.010 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:36.268 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:36.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:36.268 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:36.504 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install libcephfs2 2024-09-11T08:41:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:41:37.088 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:37.180 INFO:teuthology.orchestra.run.smithi086.stdout:Package libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:37.224 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:37.225 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:41:37.225 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:37.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install libcephfs-devel 2024-09-11T08:41:37.859 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:37.924 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:41:37.952 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T08:41:37.991 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:37.991 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repo Size 2024-09-11T08:41:37.991 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:37.992 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T08:41:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:37.992 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:37.992 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:37.992 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-11T08:41:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:37.992 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 32 k 2024-09-11T08:41:37.993 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 142 k 2024-09-11T08:41:37.993 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:38.214 INFO:teuthology.orchestra.run.smithi086.stdout:libcephfs-devel-19.1.1-298.gb53b6d3e.el9.x86_64 143 kB/s | 32 kB 00:00 2024-09-11T08:41:38.215 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:38.215 INFO:teuthology.orchestra.run.smithi086.stdout:Total 142 kB/s | 32 kB 00:00 2024-09-11T08:41:38.215 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:38.218 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:38.218 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:38.236 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:38.236 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:38.316 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:38.797 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:39.028 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:39.028 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:39.028 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:39.028 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:39.028 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:39.028 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:39.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install librados2 2024-09-11T08:41:39.759 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:39.849 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:39.892 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:39.893 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:41:39.893 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:39.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install librbd1 2024-09-11T08:41:40.531 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:40.550 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T08:41:40.550 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 1/12 2024-09-11T08:41:40.550 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:41:40.551 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:41:40.622 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:40.665 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:40.666 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:41:40.666 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:40.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install python3-rados 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:41:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:41:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:41:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:41:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:41:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:41:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:41:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:40.958 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:41.290 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:41.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:41:41.380 INFO:teuthology.orchestra.run.smithi086.stdout:Package python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:41.424 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:41.425 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:41:41.425 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:41.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install python3-rgw 2024-09-11T08:41:41.950 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:17 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:42.057 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:42.091 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:41:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout:Install 13 Packages 2024-09-11T08:41:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:42.095 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 2.2 M 2024-09-11T08:41:42.095 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 26 M 2024-09-11T08:41:42.095 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:42.148 INFO:teuthology.orchestra.run.smithi086.stdout:Package python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:42.191 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:42.192 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:41:42.192 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:42.254 INFO:teuthology.orchestra.run.smithi045.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 230 kB/s | 19 kB 00:00 2024-09-11T08:41:42.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install python3-cephfs 2024-09-11T08:41:42.271 INFO:teuthology.orchestra.run.smithi045.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 256 kB/s | 26 kB 00:00 2024-09-11T08:41:42.338 INFO:teuthology.orchestra.run.smithi045.stdout:(3/13): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el9. 295 kB/s | 49 kB 00:00 2024-09-11T08:41:42.364 INFO:teuthology.orchestra.run.smithi045.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-11T08:41:42.389 INFO:teuthology.orchestra.run.smithi045.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-11T08:41:42.414 INFO:teuthology.orchestra.run.smithi045.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-09-11T08:41:42.448 INFO:teuthology.orchestra.run.smithi045.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 548 kB/s | 32 kB 00:00 2024-09-11T08:41:42.464 INFO:teuthology.orchestra.run.smithi045.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 533 kB/s | 54 kB 00:00 2024-09-11T08:41:42.481 INFO:teuthology.orchestra.run.smithi045.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 236 kB/s | 14 kB 00:00 2024-09-11T08:41:42.507 INFO:teuthology.orchestra.run.smithi045.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 3.2 MB/s | 59 kB 00:00 2024-09-11T08:41:42.532 INFO:teuthology.orchestra.run.smithi045.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-11T08:41:42.557 INFO:teuthology.orchestra.run.smithi045.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-11T08:41:42.591 INFO:teuthology.orchestra.run.smithi045.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 8.1 MB/s | 1.0 MB 00:00 2024-09-11T08:41:42.594 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:42.594 INFO:teuthology.orchestra.run.smithi045.stdout:Total 4.4 MB/s | 2.2 MB 00:00 2024-09-11T08:41:42.659 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:42.675 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:42.675 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:42.772 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:42.773 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:42.832 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:42.923 INFO:teuthology.orchestra.run.smithi086.stdout:Package python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:42.965 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:42.967 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:41:42.967 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:43.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install python3-rbd 2024-09-11T08:41:43.284 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:43.456 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:41:43.511 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:41:43.575 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:41:43.606 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:43.611 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:41:43.653 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:41:43.696 INFO:teuthology.orchestra.run.smithi086.stdout:Package python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:41:43.739 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:43.740 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-11T08:41:43.740 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:43.760 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:41:43.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install rbd-fuse 2024-09-11T08:41:43.877 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:41:43.930 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:41:44.373 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:44.515 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:44.515 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 85 k 2024-09-11T08:41:44.516 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 231 k 2024-09-11T08:41:44.517 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:44.685 INFO:teuthology.orchestra.run.smithi086.stdout:rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 507 kB/s | 85 kB 00:00 2024-09-11T08:41:44.685 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:44.686 INFO:teuthology.orchestra.run.smithi086.stdout:Total 502 kB/s | 85 kB 00:00 2024-09-11T08:41:44.686 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:44.691 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:41:44.693 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:44.693 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:44.743 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:41:44.762 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:44.762 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:44.777 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:41:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:41:44.855 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T08:41:44.912 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:45.017 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:45.427 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:45.590 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T08:41:45.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 1/13 2024-09-11T08:41:45.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:41:45.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:41:45.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:41:45.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:41:45.590 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:41:45.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:41:45.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:41:45.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:41:45.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:41:45.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:41:45.591 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:41:45.648 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:45.648 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:45.648 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:45.648 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:45.648 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:45.648 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:45.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install rbd-mirror 2024-09-11T08:41:45.923 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:41:45.923 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:45.923 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:45.923 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:45.923 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:41:45.923 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:41:45.923 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:45.924 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:46.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:41:46.486 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:46.622 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repo Size 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:46.623 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-11T08:41:46.624 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:46.624 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 3.1 M 2024-09-11T08:41:46.624 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 13 M 2024-09-11T08:41:46.624 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:46.758 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:22 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:46.898 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:46.898 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:41:46.899 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:46.900 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:46.900 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:46.900 INFO:teuthology.orchestra.run.smithi045.stdout:Install 6 Packages 2024-09-11T08:41:46.900 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:46.900 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 7.0 M 2024-09-11T08:41:46.900 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 33 M 2024-09-11T08:41:46.901 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:46.979 INFO:teuthology.orchestra.run.smithi086.stdout:rbd-mirror-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 8.7 MB/s | 3.1 MB 00:00 2024-09-11T08:41:46.979 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:46.980 INFO:teuthology.orchestra.run.smithi086.stdout:Total 8.7 MB/s | 3.1 MB 00:00 2024-09-11T08:41:46.980 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:46.989 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:46.989 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:47.072 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:47.072 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:47.303 INFO:teuthology.orchestra.run.smithi045.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-11T08:41:47.328 INFO:teuthology.orchestra.run.smithi045.stdout:(2/6): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e.el 670 kB/s | 146 kB 00:00 2024-09-11T08:41:47.379 INFO:teuthology.orchestra.run.smithi045.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-11T08:41:47.404 INFO:teuthology.orchestra.run.smithi045.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 765 kB/s | 58 kB 00:00 2024-09-11T08:41:47.454 INFO:teuthology.orchestra.run.smithi045.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 990 kB/s | 74 kB 00:00 2024-09-11T08:41:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:47.580 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:47.605 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:47.605 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:41:47.606 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:41:47.606 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:41:47.606 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:41:47.606 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:48.646 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:48.646 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:48.647 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:48.647 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:48.696 INFO:teuthology.orchestra.run.smithi045.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.8 MB/s | 6.0 MB 00:01 2024-09-11T08:41:48.699 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:48.700 INFO:teuthology.orchestra.run.smithi045.stdout:Total 3.9 MB/s | 7.0 MB 00:01 2024-09-11T08:41:48.776 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:48.789 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:48.789 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:48.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install rbd-nbd 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:41:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:41:49.485 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:48 ago on Wed 11 Sep 2024 08:40:01 AM UTC. 2024-09-11T08:41:49.621 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-11T08:41:49.622 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:49.622 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-11T08:41:49.622 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 171 k 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 494 k 2024-09-11T08:41:49.623 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-11T08:41:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 707 kB/s | 171 kB 00:00 2024-09-11T08:41:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:Total 702 kB/s | 171 kB 00:00 2024-09-11T08:41:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-11T08:41:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-11T08:41:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-11T08:41:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-11T08:41:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-11T08:41:49.996 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:41:50.094 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-11T08:41:50.097 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:41:50.168 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:41:50.194 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T08:41:50.196 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:50.595 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:50.823 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:50.823 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:50.823 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-11T08:41:50.823 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:50.823 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:41:50.823 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-11T08:41:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T08:41:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/6 2024-09-11T08:41:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:41:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:41:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:41:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:41:51.077 DEBUG:teuthology.parallel:result is None 2024-09-11T08:41:51.263 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:51.264 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:51.516 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-fuse 2024-09-11T08:41:52.110 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:28 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:52.251 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:52.252 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:52.253 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2024-09-11T08:41:52.253 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:52.253 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 977 k 2024-09-11T08:41:52.253 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.9 M 2024-09-11T08:41:52.253 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:52.425 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 536 kB/s | 80 kB 00:00 2024-09-11T08:41:52.584 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86_6 2.9 MB/s | 897 kB 00:00 2024-09-11T08:41:52.584 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:52.584 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.9 MB/s | 977 kB 00:00 2024-09-11T08:41:52.597 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:52.619 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:52.619 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:52.901 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:53.041 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:41:53.111 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T08:41:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T08:41:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/2 2024-09-11T08:41:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:41:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:54.037 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:41:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:54.037 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:54.301 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-volume 2024-09-11T08:41:54.898 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:30 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:55.038 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:55.056 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:55.056 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 285 k 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.3 M 2024-09-11T08:41:55.057 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:55.272 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-volume-19.1.1-298.gb53b6d3e.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-11T08:41:55.273 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:55.273 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-11T08:41:55.273 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:55.282 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:55.282 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:55.314 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:55.315 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:55.548 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:55.633 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:55.650 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:55.650 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:41:55.650 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:41:55.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:56.664 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T08:41:56.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:56.664 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:56.664 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T08:41:56.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:56.664 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:56.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados-devel 2024-09-11T08:41:57.569 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:41:57.708 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:41:57.708 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 127 k 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 456 k 2024-09-11T08:41:57.709 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:41:58.187 INFO:teuthology.orchestra.run.smithi045.stdout:librados-devel-19.1.1-298.gb53b6d3e.el9.x86_64. 265 kB/s | 127 kB 00:00 2024-09-11T08:41:58.187 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:41:58.187 INFO:teuthology.orchestra.run.smithi045.stdout:Total 265 kB/s | 127 kB 00:00 2024-09-11T08:41:58.188 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:41:58.193 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:41:58.193 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:41:58.262 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:41:58.263 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:41:58.426 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:41:58.538 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:58.949 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:59.219 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:41:59.219 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:59.219 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:41:59.219 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:41:59.219 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:41:59.219 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:41:59.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs2 2024-09-11T08:42:00.074 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:00.168 INFO:teuthology.orchestra.run.smithi045.stdout:Package libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:42:00.214 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:00.215 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:42:00.215 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:00.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs-devel 2024-09-11T08:42:00.866 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:01.009 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-11T08:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:01.011 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 32 k 2024-09-11T08:42:01.011 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 142 k 2024-09-11T08:42:01.011 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:42:01.182 INFO:teuthology.orchestra.run.smithi045.stdout:libcephfs-devel-19.1.1-298.gb53b6d3e.el9.x86_64 185 kB/s | 32 kB 00:00 2024-09-11T08:42:01.183 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:42:01.183 INFO:teuthology.orchestra.run.smithi045.stdout:Total 183 kB/s | 32 kB 00:00 2024-09-11T08:42:01.183 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:42:01.187 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:42:01.187 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:42:01.210 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:42:01.210 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:42:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:42:01.433 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:01.791 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:02.028 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:02.028 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:02.028 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:42:02.028 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:42:02.028 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:02.028 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:02.191 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados2 2024-09-11T08:42:02.784 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:02.878 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:42:02.924 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:02.925 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:42:02.925 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:02.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librbd1 2024-09-11T08:42:03.589 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:03.683 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:42:03.729 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:03.730 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:42:03.730 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:03.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rados 2024-09-11T08:42:04.388 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:04.483 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:42:04.528 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:04.529 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:42:04.529 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:04.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rgw 2024-09-11T08:42:05.203 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:05.297 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:42:05.343 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:05.344 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:42:05.344 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:05.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-cephfs 2024-09-11T08:42:06.008 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:06.102 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:42:06.148 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:06.149 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:42:06.149 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:06.220 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rbd 2024-09-11T08:42:06.808 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:06.902 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T08:42:06.950 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:06.951 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-11T08:42:06.951 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:07.023 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-fuse 2024-09-11T08:42:07.615 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:07.757 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:07.757 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 85 k 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 231 k 2024-09-11T08:42:07.758 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:42:07.944 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 459 kB/s | 85 kB 00:00 2024-09-11T08:42:07.945 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:42:07.945 INFO:teuthology.orchestra.run.smithi045.stdout:Total 456 kB/s | 85 kB 00:00 2024-09-11T08:42:07.945 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:42:07.952 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:42:07.952 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:42:08.019 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:42:08.019 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:42:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:42:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:08.613 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:08.862 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:08.862 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:08.862 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:42:08.862 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:42:08.862 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:08.862 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:09.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-mirror 2024-09-11T08:42:09.723 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:09.863 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:09.864 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:09.864 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-11T08:42:09.864 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 3.1 M 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 13 M 2024-09-11T08:42:09.865 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:42:10.190 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-mirror-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 9.5 MB/s | 3.1 MB 00:00 2024-09-11T08:42:10.191 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:42:10.191 INFO:teuthology.orchestra.run.smithi045.stdout:Total 9.5 MB/s | 3.1 MB 00:00 2024-09-11T08:42:10.192 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:42:10.201 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:42:10.201 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:42:10.289 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:42:10.289 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:42:10.742 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:42:10.816 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:10.843 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:10.843 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:42:10.843 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:42:10.843 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:42:10.843 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:42:10.844 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:11.962 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:11.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:42:11.962 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:42:11.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:12.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-nbd 2024-09-11T08:42:12.805 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:48 ago on Wed 11 Sep 2024 08:40:24 AM UTC. 2024-09-11T08:42:12.946 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-11T08:42:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:12.948 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 171 k 2024-09-11T08:42:12.948 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 494 k 2024-09-11T08:42:12.948 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-11T08:42:13.136 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 905 kB/s | 171 kB 00:00 2024-09-11T08:42:13.137 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:42:13.137 INFO:teuthology.orchestra.run.smithi045.stdout:Total 898 kB/s | 171 kB 00:00 2024-09-11T08:42:13.138 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-11T08:42:13.146 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-11T08:42:13.146 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-11T08:42:13.213 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-11T08:42:13.213 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-11T08:42:13.376 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-11T08:42:13.487 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:13.872 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T08:42:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:14.135 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-11T08:42:14.135 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T08:42:14.136 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:14.136 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-11T08:42:14.366 DEBUG:teuthology.parallel:result is None 2024-09-11T08:42:14.367 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T08:42:14.519 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:42:14.541 INFO:teuthology.orchestra.run.smithi045.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T08:42:14.542 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T08:42:14.542 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T08:42:14.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T08:42:14.694 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:42:14.719 INFO:teuthology.orchestra.run.smithi086.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T08:42:14.719 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T08:42:14.720 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T08:42:14.721 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T08:42:14.722 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:14.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:42:14.748 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:14.748 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:42:14.790 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T08:42:14.790 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:14.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:42:14.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:42:14.881 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:14.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:42:14.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:42:14.983 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T08:42:14.983 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:14.983 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:42:15.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:42:15.086 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:15.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:42:15.111 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:42:15.175 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T08:42:15.175 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:15.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:42:15.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:42:15.267 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:15.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:42:15.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:42:15.358 INFO:teuthology.run_tasks:Running task ceph... 2024-09-11T08:42:15.445 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-11T08:42:15.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2024-09-11T08:42:15.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2024-09-11T08:42:15.472 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-11T08:42:15.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T08:42:15.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T08:42:15.539 INFO:tasks.ceph:Creating extra log directories... 2024-09-11T08:42:15.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T08:42:15.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T08:42:15.620 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-11T08:42:15.621 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-11T08:42:15.621 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/journal}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900665', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 6175, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'sleep_before_teardown': 0, 'subset': '88/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae', 'targets': {'smithi045.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRkVssnmE3WL0XhHzfwsWQgF3iOSXul2tX82pLnasxjkgPmS9sR8dNCLRSa9dGruGJngD/nX9d+m/WAPTLjIyk=', 'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDDxr/62T/l8T1vSahPz381O0lr2vUIZokzOVktnY22s5j317G5X/d2KH0iLYvdmtfYPiBLsgUTmZu6vnLDsJ64='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'workunit': {'clients': {'client.0': ['rbd/journal.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_21:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-11T08:42:15.622 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T08:42:15.648 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T08:42:15.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T08:42:15.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T08:42:15.731 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:15.732 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:42:15.786 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:42:15.786 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-09-11T08:42:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:42:15.842 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:42:15.843 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T08:42:15.843 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:42:15.843 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:42:15.843 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-11 08:42:11.134283607 +0000 2024-09-11T08:42:15.843 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-11 08:39:24.156876898 +0000 2024-09-11T08:42:15.843 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-11 08:39:24.156876898 +0000 2024-09-11T08:42:15.843 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-11 08:39:24.156876898 +0000 2024-09-11T08:42:15.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:42:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-11T08:42:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-11T08:42:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 8.7746e-05 s, 5.8 MB/s 2024-09-11T08:42:15.907 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:42:15.964 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-09-11T08:42:16.020 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:42:16.021 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:42:16.021 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T08:42:16.021 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:42:16.021 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:42:16.021 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-11 08:42:11.135283591 +0000 2024-09-11T08:42:16.021 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-11 08:39:24.491871493 +0000 2024-09-11T08:42:16.021 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-11 08:39:24.491871493 +0000 2024-09-11T08:42:16.021 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-11 08:39:24.491871493 +0000 2024-09-11T08:42:16.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:42:16.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-11T08:42:16.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-11T08:42:16.085 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000100805 s, 5.1 MB/s 2024-09-11T08:42:16.086 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:42:16.143 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-09-11T08:42:16.199 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:42:16.199 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:42:16.199 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T08:42:16.199 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:42:16.199 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:42:16.199 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-11 08:42:11.135283591 +0000 2024-09-11T08:42:16.200 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-11 08:39:24.816866250 +0000 2024-09-11T08:42:16.200 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-11 08:39:24.816866250 +0000 2024-09-11T08:42:16.200 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-11 08:39:24.816866250 +0000 2024-09-11T08:42:16.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:42:16.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-11T08:42:16.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-11T08:42:16.263 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.5126e-05 s, 5.4 MB/s 2024-09-11T08:42:16.264 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:42:16.321 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-09-11T08:42:16.377 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:42:16.378 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:42:16.378 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T08:42:16.378 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:42:16.378 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:42:16.378 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-11 08:42:11.136283575 +0000 2024-09-11T08:42:16.378 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-11 08:39:25.137861072 +0000 2024-09-11T08:42:16.378 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-11 08:39:25.137861072 +0000 2024-09-11T08:42:16.378 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-11 08:39:25.137861072 +0000 2024-09-11T08:42:16.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:42:16.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-11T08:42:16.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-11T08:42:16.441 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000101297 s, 5.1 MB/s 2024-09-11T08:42:16.442 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:42:16.499 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:16.499 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:16.499 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:42:16.516 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:42:16.516 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-09-11T08:42:16.572 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:42:16.572 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:42:16.572 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-11T08:42:16.573 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:42:16.573 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:42:16.573 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-11 08:41:47.867690557 +0000 2024-09-11T08:42:16.573 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-11 08:39:20.517102063 +0000 2024-09-11T08:42:16.573 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-11 08:39:20.517102063 +0000 2024-09-11T08:42:16.573 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-11 08:39:20.517102063 +0000 2024-09-11T08:42:16.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:42:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-11T08:42:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-11T08:42:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000184375 s, 2.8 MB/s 2024-09-11T08:42:16.639 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:42:16.695 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-09-11T08:42:16.751 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:42:16.751 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:42:16.751 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-11T08:42:16.751 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:42:16.752 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:42:16.752 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-11 08:41:47.868690535 +0000 2024-09-11T08:42:16.752 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-11 08:39:20.835095217 +0000 2024-09-11T08:42:16.752 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-11 08:39:20.835095217 +0000 2024-09-11T08:42:16.752 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-11 08:39:20.835095217 +0000 2024-09-11T08:42:16.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:42:16.814 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-11T08:42:16.814 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-11T08:42:16.815 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000206436 s, 2.5 MB/s 2024-09-11T08:42:16.816 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:42:16.873 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-09-11T08:42:16.928 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:42:16.929 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:42:16.929 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-11T08:42:16.929 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:42:16.929 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:42:16.929 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-11 08:41:47.868690535 +0000 2024-09-11T08:42:16.929 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-11 08:39:21.147088500 +0000 2024-09-11T08:42:16.929 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-11 08:39:21.147088500 +0000 2024-09-11T08:42:16.929 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-11 08:39:21.147088500 +0000 2024-09-11T08:42:16.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:42:16.991 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-11T08:42:16.991 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-11T08:42:16.991 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000197729 s, 2.6 MB/s 2024-09-11T08:42:16.992 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:42:17.048 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-09-11T08:42:17.103 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:42:17.103 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:42:17.104 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-11T08:42:17.104 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:42:17.104 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:42:17.104 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-11 08:41:47.868690535 +0000 2024-09-11T08:42:17.104 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-11 08:39:21.455081869 +0000 2024-09-11T08:42:17.104 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-11 08:39:21.455081869 +0000 2024-09-11T08:42:17.104 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-11 08:39:21.455081869 +0000 2024-09-11T08:42:17.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:42:17.166 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-11T08:42:17.166 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-11T08:42:17.166 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00016699 s, 3.1 MB/s 2024-09-11T08:42:17.167 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:42:17.223 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:17.224 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-11T08:42:17.224 INFO:tasks.ceph:Generating config... 2024-09-11T08:42:17.225 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-11T08:42:17.225 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-11T08:42:17.225 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-11T08:42:17.225 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-11T08:42:17.225 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-11T08:42:17.225 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-11T08:42:17.225 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-11T08:42:17.225 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-11T08:42:17.226 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-11T08:42:17.227 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-11T08:42:17.227 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-11T08:42:17.227 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-11T08:42:17.227 INFO:tasks.ceph:Setting up mon.a... 2024-09-11T08:42:17.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-11T08:42:17.286 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.keyring 2024-09-11T08:42:17.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-11T08:42:17.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T08:42:17.360 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.45'), ('mon.c', '[v2:172.21.15.45:3301,v1:172.21.15.45:6790]'), ('mon.b', '172.21.15.86')] 2024-09-11T08:42:17.361 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790],172.21.15.86', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-11T08:42:17.361 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:17.361 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T08:42:17.416 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.45 --addv c '[v2:172.21.15.45:3301,v1:172.21.15.45:6790]' --add b 172.21.15.86 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:17.504 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:17.504 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: generated fsid 75755954-09e8-48d3-b9c1-83899efaff39 2024-09-11T08:42:17.504 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-09-11T08:42:17.504 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 0 2024-09-11T08:42:17.504 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 75755954-09e8-48d3-b9c1-83899efaff39 2024-09-11T08:42:17.505 INFO:teuthology.orchestra.run.smithi045.stdout:last_changed 2024-09-11T08:42:17.505619+0000 2024-09-11T08:42:17.505 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-11T08:42:17.505619+0000 2024-09-11T08:42:17.505 INFO:teuthology.orchestra.run.smithi045.stdout:min_mon_release 17 (quincy) 2024-09-11T08:42:17.505 INFO:teuthology.orchestra.run.smithi045.stdout:election_strategy: 1 2024-09-11T08:42:17.505 INFO:teuthology.orchestra.run.smithi045.stdout:0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-11T08:42:17.505 INFO:teuthology.orchestra.run.smithi045.stdout:1: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-09-11T08:42:17.505 INFO:teuthology.orchestra.run.smithi045.stdout:2: [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] mon.c 2024-09-11T08:42:17.505 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-11T08:42:17.506 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T08:42:17.522 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 75755954-09e8-48d3-b9c1-83899efaff39... 2024-09-11T08:42:17.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T08:42:17.565 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T08:42:17.607 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-09-11T08:42:17.607 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-09-11T08:42:17.607 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T08:42:17.607 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-09-11T08:42:17.607 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.607 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-09-11T08:42:17.607 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.607 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T08:42:17.608 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T08:42:17.609 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790],172.21.15.86" 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: mon client directed command retry = 5 2024-09-11T08:42:17.610 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on pool no app = False 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: ms inject socket failures = 5000 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 75755954-09e8-48d3-b9c1-83899efaff39 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-09-11T08:42:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.612 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: bluefs allocator = hybrid 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore allocator = hybrid 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-09-11T08:42:17.613 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1/20 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1/20 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 4/10 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: osd blocked scrub grace period = 3600 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: osd max markdown count = 1000 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: osd shutdown pgref assert = True 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.614 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-09-11T08:42:17.615 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T08:42:17.616 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T08:42:17.617 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-09-11T08:42:17.618 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-09-11T08:42:17.618 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-09-11T08:42:17.618 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-09-11T08:42:17.618 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-09-11T08:42:17.618 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T08:42:17.618 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-09-11T08:42:17.618 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.618 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T08:42:17.620 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790],172.21.15.86" 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: mon client directed command retry = 5 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on pool no app = False 2024-09-11T08:42:17.621 INFO:teuthology.orchestra.run.smithi086.stdout: ms inject socket failures = 5000 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 75755954-09e8-48d3-b9c1-83899efaff39 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-09-11T08:42:17.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-09-11T08:42:17.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: bluefs allocator = hybrid 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = hybrid 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-09-11T08:42:17.624 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: osd blocked scrub grace period = 3600 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: osd max markdown count = 1000 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: osd shutdown pgref assert = True 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-09-11T08:42:17.625 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-09-11T08:42:17.626 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-09-11T08:42:17.627 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-09-11T08:42:17.628 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-09-11T08:42:17.630 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-11T08:42:17.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-11T08:42:17.725 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-11T08:42:17.725 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:17.725 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-11T08:42:17.741 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:17.741 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-11T08:42:17.797 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:42:17.797 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:17.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T08:42:17.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T08:42:17.872 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:17.872 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:17.928 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:42:17.929 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:17.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T08:42:17.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T08:42:17.964 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:17.964 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:18.020 INFO:tasks.ceph:Setting up mon nodes... 2024-09-11T08:42:18.020 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-11T08:42:18.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-11T08:42:18.073 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-11T08:42:18.076 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T08:42:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T08:42:18.143 INFO:tasks.ceph:Setting up mds nodes... 2024-09-11T08:42:18.143 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-11T08:42:18.143 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T08:42:18.185 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-11T08:42:18.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T08:42:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-11T08:42:18.252 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-11T08:42:18.252 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-11T08:42:18.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-11T08:42:18.277 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:18.277 INFO:tasks.ceph:role: osd.0 2024-09-11T08:42:18.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:42:18.278 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T08:42:18.343 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T08:42:18.375 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-11T08:42:18.377 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-11T08:42:18.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-11T08:42:18.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-11T08:42:18.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-11T08:42:18.585 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:18.585 INFO:tasks.ceph:role: osd.1 2024-09-11T08:42:18.585 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:42:18.585 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T08:42:18.650 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T08:42:18.650 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T08:42:18.650 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T08:42:18.650 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T08:42:18.651 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T08:42:18.651 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-11T08:42:18.651 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T08:42:18.651 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T08:42:18.651 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T08:42:18.651 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T08:42:18.683 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-11T08:42:18.685 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-11T08:42:18.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-11T08:42:18.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-11T08:42:18.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-11T08:42:18.894 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:18.894 INFO:tasks.ceph:role: osd.2 2024-09-11T08:42:18.894 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:42:18.894 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T08:42:18.958 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T08:42:18.959 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T08:42:18.991 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-11T08:42:18.993 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-11T08:42:18.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-11T08:42:19.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-11T08:42:19.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-11T08:42:19.202 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:19.202 INFO:tasks.ceph:role: osd.3 2024-09-11T08:42:19.202 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:42:19.202 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-11T08:42:19.268 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T08:42:19.268 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T08:42:19.268 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T08:42:19.269 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T08:42:19.269 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T08:42:19.269 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-11T08:42:19.269 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T08:42:19.269 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T08:42:19.269 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T08:42:19.269 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T08:42:19.300 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-11T08:42:19.303 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-11T08:42:19.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-11T08:42:19.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-11T08:42:19.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:19.537+0000 7fb3bf40f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-11T08:42:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:19.537+0000 7fb3bf40f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-11T08:42:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:19.537+0000 7fb3bf40f640 -1 bdev(0x55e4b3893800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-11T08:42:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:19.537+0000 7fb3bf40f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-11T08:42:22.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-11T08:42:22.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:22.836+0000 7f788f626640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-11T08:42:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:22.836+0000 7f788f626640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-11T08:42:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:22.836+0000 7f788f626640 -1 bdev(0x55f2d5793800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-11T08:42:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:22.836+0000 7f788f626640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-11T08:42:26.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-11T08:42:26.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:26.128+0000 7fbf90d58640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-11T08:42:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:26.128+0000 7fbf90d58640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-11T08:42:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:26.128+0000 7fbf90d58640 -1 bdev(0x55f37df7b800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-11T08:42:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:26.128+0000 7fbf90d58640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-11T08:42:29.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T08:42:29.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:29.663+0000 7f9711c0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-11T08:42:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:29.663+0000 7f9711c0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-11T08:42:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:29.663+0000 7f9711c0d640 -1 bdev(0x55c31d6a9800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-11T08:42:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-11T08:42:29.663+0000 7f9711c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-11T08:42:33.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-11T08:42:33.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-11T08:42:33.145 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:33.146 INFO:tasks.ceph:role: osd.4 2024-09-11T08:42:33.146 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:42:33.146 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T08:42:33.222 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T08:42:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T08:42:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T08:42:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T08:42:33.223 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T08:42:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-09-11T08:42:33.223 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T08:42:33.223 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T08:42:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T08:42:33.223 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T08:42:33.712 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-09-11T08:42:33.714 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-09-11T08:42:33.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-11T08:42:33.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-11T08:42:33.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-11T08:42:33.958 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:33.958 INFO:tasks.ceph:role: osd.5 2024-09-11T08:42:33.959 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:42:33.959 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T08:42:34.033 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T08:42:34.033 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T08:42:34.033 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T08:42:34.033 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T08:42:34.033 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T08:42:34.033 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-09-11T08:42:34.033 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T08:42:34.033 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T08:42:34.034 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T08:42:34.034 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T08:42:34.488 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-09-11T08:42:34.490 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-09-11T08:42:34.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-11T08:42:34.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-11T08:42:34.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-11T08:42:34.690 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:34.690 INFO:tasks.ceph:role: osd.6 2024-09-11T08:42:34.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:42:34.690 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T08:42:34.764 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T08:42:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T08:42:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T08:42:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T08:42:34.764 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T08:42:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-09-11T08:42:34.764 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T08:42:34.764 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T08:42:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T08:42:34.765 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T08:42:35.256 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-09-11T08:42:35.259 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-09-11T08:42:35.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-11T08:42:35.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-11T08:42:35.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-11T08:42:35.507 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T08:42:35.507 INFO:tasks.ceph:role: osd.7 2024-09-11T08:42:35.507 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:42:35.507 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-11T08:42:35.580 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T08:42:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T08:42:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T08:42:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T08:42:35.580 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T08:42:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-09-11T08:42:35.581 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T08:42:35.581 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T08:42:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T08:42:35.581 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T08:42:36.100 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-09-11T08:42:36.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-09-11T08:42:36.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-11T08:42:36.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-11T08:42:36.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:36.379+0000 7f06b5c31640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-11T08:42:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:36.379+0000 7f06b5c31640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-11T08:42:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:36.379+0000 7f06b5c31640 -1 bdev(0x5646ab85d800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-11T08:42:36.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:36.379+0000 7f06b5c31640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-11T08:42:39.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-11T08:42:39.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:39.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:39.665+0000 7f58d8b02640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-11T08:42:39.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:39.666+0000 7f58d8b02640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-11T08:42:39.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:39.666+0000 7f58d8b02640 -1 bdev(0x55f68d2a7800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-11T08:42:39.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:39.666+0000 7f58d8b02640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-11T08:42:42.826 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-11T08:42:42.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:42.949+0000 7f59630d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-11T08:42:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:42.950+0000 7f59630d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-11T08:42:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:42.950+0000 7f59630d6640 -1 bdev(0x562ae0269800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-11T08:42:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:42.950+0000 7f59630d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-11T08:42:46.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-11T08:42:46.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:46.238+0000 7f84f7e1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-11T08:42:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:46.238+0000 7f84f7e1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-11T08:42:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:46.238+0000 7f84f7e1e640 -1 bdev(0x556fbf711800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-11T08:42:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:46.238+0000 7f84f7e1e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-11T08:42:49.889 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-11T08:42:49.918 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-11T08:42:49.918 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:49.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-11T08:42:49.950 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:49.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-11T08:42:50.021 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:50.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-11T08:42:50.089 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:50.089 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-11T08:42:50.160 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:50.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-11T08:42:50.225 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:50.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-11T08:42:50.254 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:50.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-11T08:42:50.323 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:50.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-11T08:42:50.391 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:50.391 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-11T08:42:50.458 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:50.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-11T08:42:50.525 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:50.525 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-11T08:42:50.543 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:50.543 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-11T08:42:50.583 INFO:tasks.ceph:Adding keys to all mons... 2024-09-11T08:42:50.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T08:42:50.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.y] 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDqV+FmEgl6BBAA8cUX9TuXeHQNR5ATT+EmWw== 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDrV+FmAtERIBAAbmhHFfc+Mp+bBWG5d5YHMw== 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDuV+Fm+tjoMRAA6VcIhuToDodppbeGqrQVMQ== 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDyV+Fm/9WyBxAAc3JuT00VWQNvN0HqKnXq4g== 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2024-09-11T08:42:50.627 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQD1V+FmWnWjJxAAgs98k8dvfPGKFVYRTzoKCw== 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDqV+FmkO1yCBAAaTQWNzVP0vRkuhoqUCgt5A== 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQD8V+FmlW6sFhAAisUmi5gR5wMRmjxMiNbV9A== 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQD/V+FmdkO7JxAABmWTRMnqF9ksglGplskzxg== 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.6] 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQACWOFm9DypOBAANySxwH81Re1ot1REk5PxxQ== 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.7] 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQAGWOFmjlJDDhAAv5GLR8jJvMZb2kcNpJC6Uw== 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDqV+Fmb/okCxAAyaANsnlXXWX3VNaoCoBzJA== 2024-09-11T08:42:50.628 INFO:teuthology.orchestra.run.smithi045.stdout:[client.1] 2024-09-11T08:42:50.629 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDqV+FmDz1ZDhAAHsRkpAlb+bA2sJBw1VKm9A== 2024-09-11T08:42:50.647 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.y] 2024-09-11T08:42:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDqV+FmEgl6BBAA8cUX9TuXeHQNR5ATT+EmWw== 2024-09-11T08:42:50.647 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2024-09-11T08:42:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDrV+FmAtERIBAAbmhHFfc+Mp+bBWG5d5YHMw== 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDuV+Fm+tjoMRAA6VcIhuToDodppbeGqrQVMQ== 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDyV+Fm/9WyBxAAc3JuT00VWQNvN0HqKnXq4g== 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.3] 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQD1V+FmWnWjJxAAgs98k8dvfPGKFVYRTzoKCw== 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDqV+FmkO1yCBAAaTQWNzVP0vRkuhoqUCgt5A== 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.4] 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQD8V+FmlW6sFhAAisUmi5gR5wMRmjxMiNbV9A== 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.5] 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQD/V+FmdkO7JxAABmWTRMnqF9ksglGplskzxg== 2024-09-11T08:42:50.648 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.6] 2024-09-11T08:42:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQACWOFm9DypOBAANySxwH81Re1ot1REk5PxxQ== 2024-09-11T08:42:50.649 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.7] 2024-09-11T08:42:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQAGWOFmjlJDDhAAv5GLR8jJvMZb2kcNpJC6Uw== 2024-09-11T08:42:50.649 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-09-11T08:42:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDqV+Fmb/okCxAAyaANsnlXXWX3VNaoCoBzJA== 2024-09-11T08:42:50.649 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2024-09-11T08:42:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDqV+FmDz1ZDhAAHsRkpAlb+bA2sJBw1VKm9A== 2024-09-11T08:42:50.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T08:42:50.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T08:42:50.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:50.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:50.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:50.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:50.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:50.852 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:50.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:50.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:50.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T08:42:50.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T08:42:51.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:51.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:51.061 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:51.063 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:51.114 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:51.116 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:51.167 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:51.169 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T08:42:51.220 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T08:42:51.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T08:42:51.276 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T08:42:51.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T08:42:51.333 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-11T08:42:51.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-11T08:42:51.359 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T08:42:51.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-11T08:42:51.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-11T08:42:51.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T08:42:51.647 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-11T08:42:51.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-11T08:42:51.698 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T08:42:51.798 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-11T08:42:51.822 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:51.825 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T08:42:51.878 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-11T08:42:51.878 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-11T08:42:51.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-11T08:42:51.881 INFO:tasks.ceph.mon.a:Started 2024-09-11T08:42:51.881 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-11T08:42:51.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-11T08:42:51.883 INFO:tasks.ceph.mon.c:Started 2024-09-11T08:42:51.883 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-11T08:42:51.883 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-11T08:42:51.920 INFO:tasks.ceph.mon.b:Started 2024-09-11T08:42:51.920 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-11T08:42:51.920 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-11T08:42:51.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-11T08:42:51.923 INFO:tasks.ceph.mgr.y:Started 2024-09-11T08:42:51.923 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-11T08:42:51.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-11T08:42:51.925 INFO:tasks.ceph.mgr.x:Started 2024-09-11T08:42:51.925 INFO:tasks.ceph:Setting crush tunables to default 2024-09-11T08:42:51.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd crush tunables default 2024-09-11T08:42:58.114 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-09-11T08:42:58.119 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:58.120+0000 7f497c00d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:42:58.125 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:58.126+0000 7fe8cc877040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:42:58.127 INFO:tasks.ceph:check_enable_crimson: False 2024-09-11T08:42:58.127 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-11T08:42:58.128 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:58.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-11T08:42:58.154 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:58.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-11T08:42:58.162 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:58.162+0000 7f497c00d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:42:58.171 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:58.172+0000 7fe8cc877040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:42:58.198 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:58.198 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-11T08:42:58.264 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:42:58.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-11T08:42:58.301 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:58.302+0000 7f497c00d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T08:42:58.323 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:58.323+0000 7fe8cc877040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T08:42:58.331 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:58.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-11T08:42:58.353 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:58.353+0000 7f497c00d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:42:58.357 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:58.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-11T08:42:58.376 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:58.377+0000 7fe8cc877040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:42:58.422 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:58.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-11T08:42:58.487 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-11T08:42:58.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-11T08:42:58.517 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:58.517+0000 7f497c00d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:42:58.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new ac6b86cf-4ee8-49eb-a680-a350b1c9879c 0 2024-09-11T08:42:58.546 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:58.546+0000 7fe8cc877040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:42:58.632 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:58.633+0000 7f497c00d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:42:58.661 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:58.661+0000 7fe8cc877040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:42:58.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-11T08:42:58.882+0000 7fef22d9d640 0 --2- 172.21.15.86:0/2069756426 >> v2:172.21.15.86:3300/0 conn(0x7fef240ad5d0 0x7fef240ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-11T08:42:58.958 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:58.959+0000 7f497c00d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:42:58.963 INFO:teuthology.orchestra.run.smithi086.stdout:0 2024-09-11T08:42:58.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new e27d776b-f77b-4111-a536-5f5d8030c696 1 2024-09-11T08:42:58.993 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:58.993+0000 7fe8cc877040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:42:59.016 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:59.016+0000 7f497c00d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:42:59.052 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:59.052+0000 7fe8cc877040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:42:59.151 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-09-11T08:42:59.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 6b8e44f6-395b-447d-a5d3-c19a0a4be61b 2 2024-09-11T08:42:59.175 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:42:59.175+0000 7f497c00d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:42:59.210 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:42:59.210+0000 7fe8cc877040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:42:59.353 INFO:teuthology.orchestra.run.smithi086.stdout:2 2024-09-11T08:42:59.365 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 1d1e1983-4c89-4cb4-a6b4-af244f941a7a 3 2024-09-11T08:42:59.548 INFO:teuthology.orchestra.run.smithi086.stdout:3 2024-09-11T08:42:59.560 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new fd9239ee-3a74-4399-af2f-1e9a63a2f642 4 2024-09-11T08:42:59.747 INFO:teuthology.orchestra.run.smithi086.stdout:4 2024-09-11T08:42:59.758 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new cfff2555-5ad4-4629-a725-afadf9b6a96e 5 2024-09-11T08:42:59.940 INFO:teuthology.orchestra.run.smithi086.stdout:5 2024-09-11T08:42:59.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 4588b13e-af90-48a7-a10d-f7a185a902e2 6 2024-09-11T08:43:00.132 INFO:teuthology.orchestra.run.smithi086.stdout:6 2024-09-11T08:43:00.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 12da1d21-9c59-4462-9d5a-61801dab3e4d 7 2024-09-11T08:43:00.177 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:00.177+0000 7f497c00d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:43:00.212 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:00.212+0000 7fe8cc877040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:43:00.234 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:00.234+0000 7f497c00d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:43:00.267 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:00.268+0000 7fe8cc877040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:43:00.321 INFO:teuthology.orchestra.run.smithi086.stdout:7 2024-09-11T08:43:00.332 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T08:43:00.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-11T08:43:00.335 INFO:tasks.ceph.osd.0:Started 2024-09-11T08:43:00.335 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T08:43:00.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T08:43:00.337 INFO:tasks.ceph.osd.1:Started 2024-09-11T08:43:00.338 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T08:43:00.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T08:43:00.340 INFO:tasks.ceph.osd.2:Started 2024-09-11T08:43:00.340 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-11T08:43:00.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-11T08:43:00.342 INFO:tasks.ceph.osd.3:Started 2024-09-11T08:43:00.343 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-11T08:43:00.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-11T08:43:00.374 INFO:tasks.ceph.osd.4:Started 2024-09-11T08:43:00.375 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-11T08:43:00.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-11T08:43:00.377 INFO:tasks.ceph.osd.5:Started 2024-09-11T08:43:00.377 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-11T08:43:00.377 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-11T08:43:00.379 INFO:tasks.ceph.osd.6:Started 2024-09-11T08:43:00.379 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-11T08:43:00.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-11T08:43:00.382 INFO:tasks.ceph.osd.7:Started 2024-09-11T08:43:00.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T08:43:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:00.601 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:43:00.320620+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T08:43:00.320620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:43:00.612 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-11T08:43:00.612 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-11T08:43:00.640 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:00.640+0000 7f497c00d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:43:00.724 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:00.724+0000 7fe8cc877040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:43:00.725 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:00.725+0000 7f497c00d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:43:00.813 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:00.813+0000 7fe8cc877040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:43:00.837 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:00.837+0000 7f497c00d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:43:00.884 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:00.884+0000 7f497c00d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:43:00.924 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:00.925+0000 7fe8cc877040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:43:00.972 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:00.973+0000 7fe8cc877040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:43:00.997 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:00.998+0000 7f497c00d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:43:01.053 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:01.053+0000 7f497c00d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:43:01.090 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.091+0000 7fe8cc877040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:43:01.100 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:01.100+0000 7f497c00d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:43:01.145 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.146+0000 7fe8cc877040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:43:01.193 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.193+0000 7fe8cc877040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:43:01.214 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:01.215+0000 7f497c00d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:43:01.265 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:01.265+0000 7f497c00d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:43:01.310 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.310+0000 7fe8cc877040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:43:01.316 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:01.317+0000 7f497c00d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:43:01.323 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:01.323+0000 7fee2f039640 -1 Falling back to public interface 2024-09-11T08:43:01.363 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.363+0000 7fe8cc877040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:43:01.367 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:01.368+0000 7f2eefc9c640 -1 Falling back to public interface 2024-09-11T08:43:01.389 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:01.389+0000 7f497c00d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:43:01.416 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.416+0000 7fe8cc877040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:43:01.416 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T08:43:01.434 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:01.435+0000 7f497c00d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:43:01.478 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:01.479+0000 7f497c00d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:43:01.488 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.489+0000 7fe8cc877040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:43:01.537 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.537+0000 7fe8cc877040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:43:01.552 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:01.552+0000 7ff59da13640 -1 Falling back to public interface 2024-09-11T08:43:01.560 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:01.560+0000 7f8de2e12640 -1 Falling back to public interface 2024-09-11T08:43:01.580 INFO:teuthology.misc.health.smithi045.stdout: 2024-09-11T08:43:01.581 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":10,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:43:00.320620+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T08:43:00.320620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:43:01.584 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:01.584+0000 7fe8cc877040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:43:01.590 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-11T08:43:01.622 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:01.623+0000 7f2e857bb640 -1 Falling back to public interface 2024-09-11T08:43:01.626 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:01.626+0000 7f43abcfe640 -1 Falling back to public interface 2024-09-11T08:43:01.791 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:01.791+0000 7efd0560d640 -1 Falling back to public interface 2024-09-11T08:43:01.871 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:01.871+0000 7fc6b4ddb640 -1 Falling back to public interface 2024-09-11T08:43:02.141 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:02.141+0000 7f497c00d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:43:02.272 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-11T08:43:02.273+0000 7fe8cc877040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:43:03.157 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:03.157+0000 7f491bc35640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T08:43:03.157 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-11T08:43:03.157+0000 7f491bc35640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T08:43:05.119 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:05.119+0000 7fee2f039640 -1 osd.3 0 log_to_monitors true 2024-09-11T08:43:05.184 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:05.184+0000 7f2eefc9c640 -1 osd.4 0 log_to_monitors true 2024-09-11T08:43:05.346 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:05.347+0000 7ff59da13640 -1 osd.0 0 log_to_monitors true 2024-09-11T08:43:05.391 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:05.391+0000 7f8de2e12640 -1 osd.2 0 log_to_monitors true 2024-09-11T08:43:05.450 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:05.451+0000 7f2e857bb640 -1 osd.6 0 log_to_monitors true 2024-09-11T08:43:05.454 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:05.454+0000 7f43abcfe640 -1 osd.7 0 log_to_monitors true 2024-09-11T08:43:05.634 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:05.634+0000 7fc6b4ddb640 -1 osd.5 0 log_to_monitors true 2024-09-11T08:43:05.817 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:05.817+0000 7efd0560d640 -1 osd.1 0 log_to_monitors true 2024-09-11T08:43:07.069 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:07.069+0000 7fee2b7dc640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T08:43:07.081 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:07.082+0000 7fee22de8640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:43:07.118 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:07.118+0000 7f2eebc2b640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T08:43:07.127 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:07.128+0000 7f2ee3a4b640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:43:08.398 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T08:43:08.538 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:08.539+0000 7f8dde7e0640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T08:43:08.553 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:08.554+0000 7efd00f2f640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T08:43:08.557 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:08.558+0000 7f8dd6600640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:43:08.558 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:08.559+0000 7ff5992f5640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T08:43:08.569 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:08.570+0000 7efcf8d4f640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:43:08.572 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:08.573+0000 7ff591115640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:43:08.746 INFO:teuthology.misc.health.smithi045.stdout: 2024-09-11T08:43:08.746 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":14,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:43:08.334982+0000","last_up_change":"2024-09-11T08:43:07.211767+0000","last_in_change":"2024-09-11T08:43:00.320620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:43:08.201248+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6801","nonce":226138367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6803","nonce":226138367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6807","nonce":226138367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6805","nonce":226138367}]},"public_addr":"172.21.15.45:6801/226138367","cluster_addr":"172.21.15.45:6803/226138367","heartbeat_back_addr":"172.21.15.45:6807/226138367","heartbeat_front_addr":"172.21.15.45:6805/226138367","state":["exists","up"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6801","nonce":374760845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6803","nonce":374760845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6807","nonce":374760845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6805","nonce":374760845}]},"public_addr":"172.21.15.86:6801/374760845","cluster_addr":"172.21.15.86:6803/374760845","heartbeat_back_addr":"172.21.15.86:6807/374760845","heartbeat_front_addr":"172.21.15.86:6805/374760845","state":["exists","up"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.147591+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:07.119248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:43:08.757 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-09-11T08:43:08.966 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:08.965+0000 7fc6b0d6a640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T08:43:08.978 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:08.978+0000 7fc6a8b8a640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:43:09.000 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:08.999+0000 7f2e8174a640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T08:43:09.022 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:09.021+0000 7f43a7c8d640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T08:43:09.023 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:09.023+0000 7f2e7956a640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:43:09.035 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:09.035+0000 7f439faad640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:43:15.565 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T08:43:15.860 INFO:teuthology.misc.health.smithi045.stdout: 2024-09-11T08:43:15.860 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":17,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:43:11.356414+0000","last_up_change":"2024-09-11T08:43:09.343672+0000","last_in_change":"2024-09-11T08:43:00.320620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:43:08.201248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6809","nonce":2540486849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6811","nonce":2540486849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6815","nonce":2540486849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6813","nonce":2540486849}]},"public_addr":"172.21.15.45:6809/2540486849","cluster_addr":"172.21.15.45:6811/2540486849","heartbeat_back_addr":"172.21.15.45:6815/2540486849","heartbeat_front_addr":"172.21.15.45:6813/2540486849","state":["exists","up"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6825","nonce":1040844445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6827","nonce":1040844445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6831","nonce":1040844445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6829","nonce":1040844445}]},"public_addr":"172.21.15.45:6825/1040844445","cluster_addr":"172.21.15.45:6827/1040844445","heartbeat_back_addr":"172.21.15.45:6831/1040844445","heartbeat_front_addr":"172.21.15.45:6829/1040844445","state":["exists","up"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6817","nonce":1231172466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6819","nonce":1231172466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6823","nonce":1231172466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6821","nonce":1231172466}]},"public_addr":"172.21.15.45:6817/1231172466","cluster_addr":"172.21.15.45:6819/1231172466","heartbeat_back_addr":"172.21.15.45:6823/1231172466","heartbeat_front_addr":"172.21.15.45:6821/1231172466","state":["exists","up"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6801","nonce":226138367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6803","nonce":226138367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6807","nonce":226138367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6805","nonce":226138367}]},"public_addr":"172.21.15.45:6801/226138367","cluster_addr":"172.21.15.45:6803/226138367","heartbeat_back_addr":"172.21.15.45:6807/226138367","heartbeat_front_addr":"172.21.15.45:6805/226138367","state":["exists","up"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6801","nonce":374760845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6803","nonce":374760845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6807","nonce":374760845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6805","nonce":374760845}]},"public_addr":"172.21.15.86:6801/374760845","cluster_addr":"172.21.15.86:6803/374760845","heartbeat_back_addr":"172.21.15.86:6807/374760845","heartbeat_front_addr":"172.21.15.86:6805/374760845","state":["exists","up"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6825","nonce":498386186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6827","nonce":498386186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6831","nonce":498386186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6829","nonce":498386186}]},"public_addr":"172.21.15.86:6825/498386186","cluster_addr":"172.21.15.86:6827/498386186","heartbeat_back_addr":"172.21.15.86:6831/498386186","heartbeat_front_addr":"172.21.15.86:6829/498386186","state":["exists","up"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6809","nonce":1261869816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6811","nonce":1261869816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6815","nonce":1261869816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6813","nonce":1261869816}]},"public_addr":"172.21.15.86:6809/1261869816","cluster_addr":"172.21.15.86:6811/1261869816","heartbeat_back_addr":"172.21.15.86:6815/1261869816","heartbeat_front_addr":"172.21.15.86:6813/1261869816","state":["exists","up"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6817","nonce":2975777507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6819","nonce":2975777507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6823","nonce":2975777507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6821","nonce":2975777507}]},"public_addr":"172.21.15.86:6817/2975777507","cluster_addr":"172.21.15.86:6819/2975777507","heartbeat_back_addr":"172.21.15.86:6823/2975777507","heartbeat_front_addr":"172.21.15.86:6821/2975777507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.394370+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.813228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.425286+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.147591+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:07.119248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.653838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.459308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.492589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:43:15.871 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-11T08:43:15.871 INFO:tasks.ceph:Creating RBD pool 2024-09-11T08:43:15.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-11T08:43:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'rbd' created 2024-09-11T08:43:16.388 DEBUG:teuthology.orchestra.run.smithi045:> rbd --cluster ceph pool init rbd 2024-09-11T08:43:19.399 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-11T08:43:19.400 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T08:43:19.400 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-11T08:43:19.400 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-11T08:43:19.401 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-11T08:43:19.758 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:19.768 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3611418475},{"type":"v1","addr":"172.21.15.86:6833","nonce":3611418475}]},"active_addr":"172.21.15.86:6833/3611418475","active_change":"2024-09-11T08:43:02.144225+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1309191237}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1736095225}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1160345166}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":467079989}]}]} 2024-09-11T08:43:19.772 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-11T08:43:19.772 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T08:43:19.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T08:43:20.063 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:20.064 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:43:19.384410+0000","last_up_change":"2024-09-11T08:43:09.343672+0000","last_in_change":"2024-09-11T08:43:00.320620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:43:08.201248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T08:43:16.189979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6809","nonce":2540486849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6811","nonce":2540486849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6815","nonce":2540486849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6813","nonce":2540486849}]},"public_addr":"172.21.15.45:6809/2540486849","cluster_addr":"172.21.15.45:6811/2540486849","heartbeat_back_addr":"172.21.15.45:6815/2540486849","heartbeat_front_addr":"172.21.15.45:6813/2540486849","state":["exists","up"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6825","nonce":1040844445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6827","nonce":1040844445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6831","nonce":1040844445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6829","nonce":1040844445}]},"public_addr":"172.21.15.45:6825/1040844445","cluster_addr":"172.21.15.45:6827/1040844445","heartbeat_back_addr":"172.21.15.45:6831/1040844445","heartbeat_front_addr":"172.21.15.45:6829/1040844445","state":["exists","up"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6817","nonce":1231172466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6819","nonce":1231172466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6823","nonce":1231172466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6821","nonce":1231172466}]},"public_addr":"172.21.15.45:6817/1231172466","cluster_addr":"172.21.15.45:6819/1231172466","heartbeat_back_addr":"172.21.15.45:6823/1231172466","heartbeat_front_addr":"172.21.15.45:6821/1231172466","state":["exists","up"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6801","nonce":226138367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6803","nonce":226138367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6807","nonce":226138367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6805","nonce":226138367}]},"public_addr":"172.21.15.45:6801/226138367","cluster_addr":"172.21.15.45:6803/226138367","heartbeat_back_addr":"172.21.15.45:6807/226138367","heartbeat_front_addr":"172.21.15.45:6805/226138367","state":["exists","up"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6801","nonce":374760845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6803","nonce":374760845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6807","nonce":374760845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6805","nonce":374760845}]},"public_addr":"172.21.15.86:6801/374760845","cluster_addr":"172.21.15.86:6803/374760845","heartbeat_back_addr":"172.21.15.86:6807/374760845","heartbeat_front_addr":"172.21.15.86:6805/374760845","state":["exists","up"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6825","nonce":498386186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6827","nonce":498386186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6831","nonce":498386186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6829","nonce":498386186}]},"public_addr":"172.21.15.86:6825/498386186","cluster_addr":"172.21.15.86:6827/498386186","heartbeat_back_addr":"172.21.15.86:6831/498386186","heartbeat_front_addr":"172.21.15.86:6829/498386186","state":["exists","up"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6809","nonce":1261869816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6811","nonce":1261869816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6815","nonce":1261869816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6813","nonce":1261869816}]},"public_addr":"172.21.15.86:6809/1261869816","cluster_addr":"172.21.15.86:6811/1261869816","heartbeat_back_addr":"172.21.15.86:6815/1261869816","heartbeat_front_addr":"172.21.15.86:6813/1261869816","state":["exists","up"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6817","nonce":2975777507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6819","nonce":2975777507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6823","nonce":2975777507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6821","nonce":2975777507}]},"public_addr":"172.21.15.86:6817/2975777507","cluster_addr":"172.21.15.86:6819/2975777507","heartbeat_back_addr":"172.21.15.86:6823/2975777507","heartbeat_front_addr":"172.21.15.86:6821/2975777507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.394370+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.813228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.425286+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.147591+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:07.119248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.653838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.459308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.492589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:43:20.077 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T08:43:20.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T08:43:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:20.370 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:43:19.384410+0000","last_up_change":"2024-09-11T08:43:09.343672+0000","last_in_change":"2024-09-11T08:43:00.320620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:43:08.201248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T08:43:16.189979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6809","nonce":2540486849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6811","nonce":2540486849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6815","nonce":2540486849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6813","nonce":2540486849}]},"public_addr":"172.21.15.45:6809/2540486849","cluster_addr":"172.21.15.45:6811/2540486849","heartbeat_back_addr":"172.21.15.45:6815/2540486849","heartbeat_front_addr":"172.21.15.45:6813/2540486849","state":["exists","up"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6825","nonce":1040844445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6827","nonce":1040844445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6831","nonce":1040844445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6829","nonce":1040844445}]},"public_addr":"172.21.15.45:6825/1040844445","cluster_addr":"172.21.15.45:6827/1040844445","heartbeat_back_addr":"172.21.15.45:6831/1040844445","heartbeat_front_addr":"172.21.15.45:6829/1040844445","state":["exists","up"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6817","nonce":1231172466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6819","nonce":1231172466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6823","nonce":1231172466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6821","nonce":1231172466}]},"public_addr":"172.21.15.45:6817/1231172466","cluster_addr":"172.21.15.45:6819/1231172466","heartbeat_back_addr":"172.21.15.45:6823/1231172466","heartbeat_front_addr":"172.21.15.45:6821/1231172466","state":["exists","up"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6801","nonce":226138367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6803","nonce":226138367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6807","nonce":226138367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6805","nonce":226138367}]},"public_addr":"172.21.15.45:6801/226138367","cluster_addr":"172.21.15.45:6803/226138367","heartbeat_back_addr":"172.21.15.45:6807/226138367","heartbeat_front_addr":"172.21.15.45:6805/226138367","state":["exists","up"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6801","nonce":374760845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6803","nonce":374760845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6807","nonce":374760845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6805","nonce":374760845}]},"public_addr":"172.21.15.86:6801/374760845","cluster_addr":"172.21.15.86:6803/374760845","heartbeat_back_addr":"172.21.15.86:6807/374760845","heartbeat_front_addr":"172.21.15.86:6805/374760845","state":["exists","up"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6825","nonce":498386186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6827","nonce":498386186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6831","nonce":498386186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6829","nonce":498386186}]},"public_addr":"172.21.15.86:6825/498386186","cluster_addr":"172.21.15.86:6827/498386186","heartbeat_back_addr":"172.21.15.86:6831/498386186","heartbeat_front_addr":"172.21.15.86:6829/498386186","state":["exists","up"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6809","nonce":1261869816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6811","nonce":1261869816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6815","nonce":1261869816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6813","nonce":1261869816}]},"public_addr":"172.21.15.86:6809/1261869816","cluster_addr":"172.21.15.86:6811/1261869816","heartbeat_back_addr":"172.21.15.86:6815/1261869816","heartbeat_front_addr":"172.21.15.86:6813/1261869816","state":["exists","up"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6817","nonce":2975777507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6819","nonce":2975777507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6823","nonce":2975777507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6821","nonce":2975777507}]},"public_addr":"172.21.15.86:6817/2975777507","cluster_addr":"172.21.15.86:6819/2975777507","heartbeat_back_addr":"172.21.15.86:6823/2975777507","heartbeat_front_addr":"172.21.15.86:6821/2975777507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.394370+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.813228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.425286+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.147591+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:07.119248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.653838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.459308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.492589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:43:20.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T08:43:20.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T08:43:20.384 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T08:43:20.384 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-11T08:43:20.384 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-11T08:43:20.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-11T08:43:20.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-11T08:43:20.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-11T08:43:20.627 INFO:teuthology.orchestra.run.smithi045.stdout:64424509444 2024-09-11T08:43:20.627 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T08:43:20.638 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-09-11T08:43:20.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T08:43:20.645 INFO:teuthology.orchestra.run.smithi045.stdout:64424509445 2024-09-11T08:43:20.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-11T08:43:20.646 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-09-11T08:43:20.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T08:43:20.647 INFO:teuthology.orchestra.run.smithi045.stdout:55834574853 2024-09-11T08:43:20.647 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T08:43:20.648 INFO:teuthology.orchestra.run.smithi045.stdout:64424509444 2024-09-11T08:43:20.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T08:43:20.649 INFO:teuthology.orchestra.run.smithi045.stdout:55834574852 2024-09-11T08:43:20.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T08:43:20.650 INFO:teuthology.orchestra.run.smithi045.stdout:64424509444 2024-09-11T08:43:20.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-11T08:43:21.208 INFO:teuthology.orchestra.run.smithi045.stdout:64424509442 2024-09-11T08:43:21.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509442 for osd.0 2024-09-11T08:43:21.252 INFO:teuthology.orchestra.run.smithi045.stdout:64424509442 2024-09-11T08:43:21.266 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509442 for osd.7 2024-09-11T08:43:21.289 INFO:teuthology.orchestra.run.smithi045.stdout:55834574852 2024-09-11T08:43:21.297 INFO:teuthology.orchestra.run.smithi045.stdout:64424509443 2024-09-11T08:43:21.298 INFO:teuthology.orchestra.run.smithi045.stdout:55834574851 2024-09-11T08:43:21.300 INFO:teuthology.orchestra.run.smithi045.stdout:64424509444 2024-09-11T08:43:21.304 INFO:teuthology.orchestra.run.smithi045.stdout:64424509442 2024-09-11T08:43:21.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-09-11T08:43:21.312 INFO:teuthology.orchestra.run.smithi045.stdout:64424509444 2024-09-11T08:43:21.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-09-11T08:43:21.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.6 2024-09-11T08:43:21.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.1 2024-09-11T08:43:21.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509442 for osd.5 2024-09-11T08:43:21.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.2 2024-09-11T08:43:22.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T08:43:22.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-11T08:43:22.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T08:43:22.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T08:43:22.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-11T08:43:22.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T08:43:22.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T08:43:22.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T08:43:22.760 INFO:teuthology.orchestra.run.smithi045.stdout:64424509444 2024-09-11T08:43:22.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.0 2024-09-11T08:43:22.780 DEBUG:teuthology.parallel:result is None 2024-09-11T08:43:22.833 INFO:teuthology.orchestra.run.smithi045.stdout:55834574852 2024-09-11T08:43:22.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-09-11T08:43:22.847 DEBUG:teuthology.parallel:result is None 2024-09-11T08:43:22.857 INFO:teuthology.orchestra.run.smithi045.stdout:64424509444 2024-09-11T08:43:22.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.7 2024-09-11T08:43:22.872 DEBUG:teuthology.parallel:result is None 2024-09-11T08:43:22.901 INFO:teuthology.orchestra.run.smithi045.stdout:55834574853 2024-09-11T08:43:22.909 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-09-11T08:43:22.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-11T08:43:22.919 DEBUG:teuthology.parallel:result is None 2024-09-11T08:43:22.920 INFO:teuthology.orchestra.run.smithi045.stdout:64424509445 2024-09-11T08:43:22.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-11T08:43:22.923 DEBUG:teuthology.parallel:result is None 2024-09-11T08:43:22.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.6 2024-09-11T08:43:22.938 DEBUG:teuthology.parallel:result is None 2024-09-11T08:43:22.946 INFO:teuthology.orchestra.run.smithi045.stdout:64424509444 2024-09-11T08:43:22.953 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-09-11T08:43:22.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.5 2024-09-11T08:43:22.959 DEBUG:teuthology.parallel:result is None 2024-09-11T08:43:22.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-09-11T08:43:22.966 DEBUG:teuthology.parallel:result is None 2024-09-11T08:43:22.966 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T08:43:22.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:43:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:23.296 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:43:23.310 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-11T08:43:22.152588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":239848,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734872,"statfs":{"total":773094113280,"available":772848508928,"internally_reserved":0,"allocated":2854912,"data_stored":1743702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.775424"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626333+0000","last_change":"2024-09-11T08:43:19.626333+0000","last_active":"2024-09-11T08:43:19.626333+0000","last_peered":"2024-09-11T08:43:19.626333+0000","last_clean":"2024-09-11T08:43:19.626333+0000","last_became_active":"2024-09-11T08:43:17.384965+0000","last_became_peered":"2024-09-11T08:43:17.384965+0000","last_unstale":"2024-09-11T08:43:19.626333+0000","last_undegraded":"2024-09-11T08:43:19.626333+0000","last_fullsized":"2024-09-11T08:43:19.626333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:09.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037094500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.396232+0000","last_change":"2024-09-11T08:43:19.396232+0000","last_active":"2024-09-11T08:43:19.396232+0000","last_peered":"2024-09-11T08:43:19.396232+0000","last_clean":"2024-09-11T08:43:19.396232+0000","last_became_active":"2024-09-11T08:43:17.388736+0000","last_became_peered":"2024-09-11T08:43:17.388736+0000","last_unstale":"2024-09-11T08:43:19.396232+0000","last_undegraded":"2024-09-11T08:43:19.396232+0000","last_fullsized":"2024-09-11T08:43:19.396232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:26:41.299861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051427600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626282+0000","last_change":"2024-09-11T08:43:19.626282+0000","last_active":"2024-09-11T08:43:19.626282+0000","last_peered":"2024-09-11T08:43:19.626282+0000","last_clean":"2024-09-11T08:43:19.626282+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:19.626282+0000","last_undegraded":"2024-09-11T08:43:19.626282+0000","last_fullsized":"2024-09-11T08:43:19.626282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:10.938383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049065999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.396159+0000","last_change":"2024-09-11T08:43:19.396159+0000","last_active":"2024-09-11T08:43:19.396159+0000","last_peered":"2024-09-11T08:43:19.396159+0000","last_clean":"2024-09-11T08:43:19.396159+0000","last_became_active":"2024-09-11T08:43:17.384040+0000","last_became_peered":"2024-09-11T08:43:17.384040+0000","last_unstale":"2024-09-11T08:43:19.396159+0000","last_undegraded":"2024-09-11T08:43:19.396159+0000","last_fullsized":"2024-09-11T08:43:19.396159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:18:33.305374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049366500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.395345+0000","last_change":"2024-09-11T08:43:19.394642+0000","last_active":"2024-09-11T08:43:19.395345+0000","last_peered":"2024-09-11T08:43:19.395345+0000","last_clean":"2024-09-11T08:43:19.395345+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:19.395345+0000","last_undegraded":"2024-09-11T08:43:19.395345+0000","last_fullsized":"2024-09-11T08:43:19.395345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:24.191731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010925100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.469399+0000","last_change":"2024-09-11T08:43:19.469399+0000","last_active":"2024-09-11T08:43:19.469399+0000","last_peered":"2024-09-11T08:43:19.469399+0000","last_clean":"2024-09-11T08:43:19.469399+0000","last_became_active":"2024-09-11T08:43:17.383480+0000","last_became_peered":"2024-09-11T08:43:17.383480+0000","last_unstale":"2024-09-11T08:43:19.469399+0000","last_undegraded":"2024-09-11T08:43:19.469399+0000","last_fullsized":"2024-09-11T08:43:19.469399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:43.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040896800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626257+0000","last_change":"2024-09-11T08:43:19.626257+0000","last_active":"2024-09-11T08:43:19.626257+0000","last_peered":"2024-09-11T08:43:19.626257+0000","last_clean":"2024-09-11T08:43:19.626257+0000","last_became_active":"2024-09-11T08:43:17.383933+0000","last_became_peered":"2024-09-11T08:43:17.383933+0000","last_unstale":"2024-09-11T08:43:19.626257+0000","last_undegraded":"2024-09-11T08:43:19.626257+0000","last_fullsized":"2024-09-11T08:43:19.626257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:58.667236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048316400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.394034+0000","last_change":"2024-09-11T08:43:19.394034+0000","last_active":"2024-09-11T08:43:19.394034+0000","last_peered":"2024-09-11T08:43:19.394034+0000","last_clean":"2024-09-11T08:43:19.394034+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:43:19.394034+0000","last_undegraded":"2024-09-11T08:43:19.394034+0000","last_fullsized":"2024-09-11T08:43:19.394034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:54.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719799999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.625706+0000","last_change":"2024-09-11T08:43:10.406953+0000","last_active":"2024-09-11T08:43:19.625706+0000","last_peered":"2024-09-11T08:43:19.625706+0000","last_clean":"2024-09-11T08:43:19.625706+0000","last_became_active":"2024-09-11T08:43:10.405743+0000","last_became_peered":"2024-09-11T08:43:10.405743+0000","last_unstale":"2024-09-11T08:43:19.625706+0000","last_undegraded":"2024-09-11T08:43:19.625706+0000","last_fullsized":"2024-09-11T08:43:19.625706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:41.277131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":802816,"data_stored":661463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35552,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336288,"statfs":{"total":96636764160,"available":96600358912,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:23.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:43:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:43:23.611 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-11T08:43:22.152588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":239848,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734872,"statfs":{"total":773094113280,"available":772848508928,"internally_reserved":0,"allocated":2854912,"data_stored":1743702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.775424"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626333+0000","last_change":"2024-09-11T08:43:19.626333+0000","last_active":"2024-09-11T08:43:19.626333+0000","last_peered":"2024-09-11T08:43:19.626333+0000","last_clean":"2024-09-11T08:43:19.626333+0000","last_became_active":"2024-09-11T08:43:17.384965+0000","last_became_peered":"2024-09-11T08:43:17.384965+0000","last_unstale":"2024-09-11T08:43:19.626333+0000","last_undegraded":"2024-09-11T08:43:19.626333+0000","last_fullsized":"2024-09-11T08:43:19.626333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:09.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037094500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.396232+0000","last_change":"2024-09-11T08:43:19.396232+0000","last_active":"2024-09-11T08:43:19.396232+0000","last_peered":"2024-09-11T08:43:19.396232+0000","last_clean":"2024-09-11T08:43:19.396232+0000","last_became_active":"2024-09-11T08:43:17.388736+0000","last_became_peered":"2024-09-11T08:43:17.388736+0000","last_unstale":"2024-09-11T08:43:19.396232+0000","last_undegraded":"2024-09-11T08:43:19.396232+0000","last_fullsized":"2024-09-11T08:43:19.396232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:26:41.299861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051427600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626282+0000","last_change":"2024-09-11T08:43:19.626282+0000","last_active":"2024-09-11T08:43:19.626282+0000","last_peered":"2024-09-11T08:43:19.626282+0000","last_clean":"2024-09-11T08:43:19.626282+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:19.626282+0000","last_undegraded":"2024-09-11T08:43:19.626282+0000","last_fullsized":"2024-09-11T08:43:19.626282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:10.938383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049065999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.396159+0000","last_change":"2024-09-11T08:43:19.396159+0000","last_active":"2024-09-11T08:43:19.396159+0000","last_peered":"2024-09-11T08:43:19.396159+0000","last_clean":"2024-09-11T08:43:19.396159+0000","last_became_active":"2024-09-11T08:43:17.384040+0000","last_became_peered":"2024-09-11T08:43:17.384040+0000","last_unstale":"2024-09-11T08:43:19.396159+0000","last_undegraded":"2024-09-11T08:43:19.396159+0000","last_fullsized":"2024-09-11T08:43:19.396159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:18:33.305374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049366500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.395345+0000","last_change":"2024-09-11T08:43:19.394642+0000","last_active":"2024-09-11T08:43:19.395345+0000","last_peered":"2024-09-11T08:43:19.395345+0000","last_clean":"2024-09-11T08:43:19.395345+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:19.395345+0000","last_undegraded":"2024-09-11T08:43:19.395345+0000","last_fullsized":"2024-09-11T08:43:19.395345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:24.191731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010925100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.469399+0000","last_change":"2024-09-11T08:43:19.469399+0000","last_active":"2024-09-11T08:43:19.469399+0000","last_peered":"2024-09-11T08:43:19.469399+0000","last_clean":"2024-09-11T08:43:19.469399+0000","last_became_active":"2024-09-11T08:43:17.383480+0000","last_became_peered":"2024-09-11T08:43:17.383480+0000","last_unstale":"2024-09-11T08:43:19.469399+0000","last_undegraded":"2024-09-11T08:43:19.469399+0000","last_fullsized":"2024-09-11T08:43:19.469399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:43.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040896800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626257+0000","last_change":"2024-09-11T08:43:19.626257+0000","last_active":"2024-09-11T08:43:19.626257+0000","last_peered":"2024-09-11T08:43:19.626257+0000","last_clean":"2024-09-11T08:43:19.626257+0000","last_became_active":"2024-09-11T08:43:17.383933+0000","last_became_peered":"2024-09-11T08:43:17.383933+0000","last_unstale":"2024-09-11T08:43:19.626257+0000","last_undegraded":"2024-09-11T08:43:19.626257+0000","last_fullsized":"2024-09-11T08:43:19.626257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:58.667236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048316400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.394034+0000","last_change":"2024-09-11T08:43:19.394034+0000","last_active":"2024-09-11T08:43:19.394034+0000","last_peered":"2024-09-11T08:43:19.394034+0000","last_clean":"2024-09-11T08:43:19.394034+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:43:19.394034+0000","last_undegraded":"2024-09-11T08:43:19.394034+0000","last_fullsized":"2024-09-11T08:43:19.394034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:54.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719799999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.625706+0000","last_change":"2024-09-11T08:43:10.406953+0000","last_active":"2024-09-11T08:43:19.625706+0000","last_peered":"2024-09-11T08:43:19.625706+0000","last_clean":"2024-09-11T08:43:19.625706+0000","last_became_active":"2024-09-11T08:43:10.405743+0000","last_became_peered":"2024-09-11T08:43:10.405743+0000","last_unstale":"2024-09-11T08:43:19.625706+0000","last_undegraded":"2024-09-11T08:43:19.625706+0000","last_fullsized":"2024-09-11T08:43:19.625706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:41.277131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":802816,"data_stored":661463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35552,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336288,"statfs":{"total":96636764160,"available":96600358912,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:23.612 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T08:43:23.612 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T08:43:23.612 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-11T08:43:23.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-11T08:43:23.941 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:23.941 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T08:43:23.954 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-11T08:43:23.954 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-11T08:43:23.994 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-11T08:43:23.995 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae'}} 2024-09-11T08:43:23.995 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-11T08:43:23.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-11T08:43:24.305 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-11T08:43:24.305 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T08:43:24.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:43:24.596 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:43:24.607 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-11T08:43:24.152956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":239848,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734872,"statfs":{"total":773094113280,"available":772848508928,"internally_reserved":0,"allocated":2854912,"data_stored":1743702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.777085"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626333+0000","last_change":"2024-09-11T08:43:19.626333+0000","last_active":"2024-09-11T08:43:19.626333+0000","last_peered":"2024-09-11T08:43:19.626333+0000","last_clean":"2024-09-11T08:43:19.626333+0000","last_became_active":"2024-09-11T08:43:17.384965+0000","last_became_peered":"2024-09-11T08:43:17.384965+0000","last_unstale":"2024-09-11T08:43:19.626333+0000","last_undegraded":"2024-09-11T08:43:19.626333+0000","last_fullsized":"2024-09-11T08:43:19.626333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:09.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037094500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.396232+0000","last_change":"2024-09-11T08:43:19.396232+0000","last_active":"2024-09-11T08:43:19.396232+0000","last_peered":"2024-09-11T08:43:19.396232+0000","last_clean":"2024-09-11T08:43:19.396232+0000","last_became_active":"2024-09-11T08:43:17.388736+0000","last_became_peered":"2024-09-11T08:43:17.388736+0000","last_unstale":"2024-09-11T08:43:19.396232+0000","last_undegraded":"2024-09-11T08:43:19.396232+0000","last_fullsized":"2024-09-11T08:43:19.396232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:26:41.299861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051427600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626282+0000","last_change":"2024-09-11T08:43:19.626282+0000","last_active":"2024-09-11T08:43:19.626282+0000","last_peered":"2024-09-11T08:43:19.626282+0000","last_clean":"2024-09-11T08:43:19.626282+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:19.626282+0000","last_undegraded":"2024-09-11T08:43:19.626282+0000","last_fullsized":"2024-09-11T08:43:19.626282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:10.938383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049065999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.396159+0000","last_change":"2024-09-11T08:43:19.396159+0000","last_active":"2024-09-11T08:43:19.396159+0000","last_peered":"2024-09-11T08:43:19.396159+0000","last_clean":"2024-09-11T08:43:19.396159+0000","last_became_active":"2024-09-11T08:43:17.384040+0000","last_became_peered":"2024-09-11T08:43:17.384040+0000","last_unstale":"2024-09-11T08:43:19.396159+0000","last_undegraded":"2024-09-11T08:43:19.396159+0000","last_fullsized":"2024-09-11T08:43:19.396159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:18:33.305374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049366500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.395345+0000","last_change":"2024-09-11T08:43:19.394642+0000","last_active":"2024-09-11T08:43:19.395345+0000","last_peered":"2024-09-11T08:43:19.395345+0000","last_clean":"2024-09-11T08:43:19.395345+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:19.395345+0000","last_undegraded":"2024-09-11T08:43:19.395345+0000","last_fullsized":"2024-09-11T08:43:19.395345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:24.191731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010925100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.469399+0000","last_change":"2024-09-11T08:43:19.469399+0000","last_active":"2024-09-11T08:43:19.469399+0000","last_peered":"2024-09-11T08:43:19.469399+0000","last_clean":"2024-09-11T08:43:19.469399+0000","last_became_active":"2024-09-11T08:43:17.383480+0000","last_became_peered":"2024-09-11T08:43:17.383480+0000","last_unstale":"2024-09-11T08:43:19.469399+0000","last_undegraded":"2024-09-11T08:43:19.469399+0000","last_fullsized":"2024-09-11T08:43:19.469399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:43.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040896800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626257+0000","last_change":"2024-09-11T08:43:19.626257+0000","last_active":"2024-09-11T08:43:19.626257+0000","last_peered":"2024-09-11T08:43:19.626257+0000","last_clean":"2024-09-11T08:43:19.626257+0000","last_became_active":"2024-09-11T08:43:17.383933+0000","last_became_peered":"2024-09-11T08:43:17.383933+0000","last_unstale":"2024-09-11T08:43:19.626257+0000","last_undegraded":"2024-09-11T08:43:19.626257+0000","last_fullsized":"2024-09-11T08:43:19.626257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:58.667236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048316400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.394034+0000","last_change":"2024-09-11T08:43:19.394034+0000","last_active":"2024-09-11T08:43:19.394034+0000","last_peered":"2024-09-11T08:43:19.394034+0000","last_clean":"2024-09-11T08:43:19.394034+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:43:19.394034+0000","last_undegraded":"2024-09-11T08:43:19.394034+0000","last_fullsized":"2024-09-11T08:43:19.394034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:54.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719799999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.625706+0000","last_change":"2024-09-11T08:43:10.406953+0000","last_active":"2024-09-11T08:43:19.625706+0000","last_peered":"2024-09-11T08:43:19.625706+0000","last_clean":"2024-09-11T08:43:19.625706+0000","last_became_active":"2024-09-11T08:43:10.405743+0000","last_became_peered":"2024-09-11T08:43:10.405743+0000","last_unstale":"2024-09-11T08:43:19.625706+0000","last_undegraded":"2024-09-11T08:43:19.625706+0000","last_fullsized":"2024-09-11T08:43:19.625706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:41.277131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":802816,"data_stored":661463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35552,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336288,"statfs":{"total":96636764160,"available":96600358912,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:24.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:43:24.897 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:24.897 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:43:24.909 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-11T08:43:24.152956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":239848,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734872,"statfs":{"total":773094113280,"available":772848508928,"internally_reserved":0,"allocated":2854912,"data_stored":1743702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.777085"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626333+0000","last_change":"2024-09-11T08:43:19.626333+0000","last_active":"2024-09-11T08:43:19.626333+0000","last_peered":"2024-09-11T08:43:19.626333+0000","last_clean":"2024-09-11T08:43:19.626333+0000","last_became_active":"2024-09-11T08:43:17.384965+0000","last_became_peered":"2024-09-11T08:43:17.384965+0000","last_unstale":"2024-09-11T08:43:19.626333+0000","last_undegraded":"2024-09-11T08:43:19.626333+0000","last_fullsized":"2024-09-11T08:43:19.626333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:09.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037094500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.396232+0000","last_change":"2024-09-11T08:43:19.396232+0000","last_active":"2024-09-11T08:43:19.396232+0000","last_peered":"2024-09-11T08:43:19.396232+0000","last_clean":"2024-09-11T08:43:19.396232+0000","last_became_active":"2024-09-11T08:43:17.388736+0000","last_became_peered":"2024-09-11T08:43:17.388736+0000","last_unstale":"2024-09-11T08:43:19.396232+0000","last_undegraded":"2024-09-11T08:43:19.396232+0000","last_fullsized":"2024-09-11T08:43:19.396232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:26:41.299861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051427600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626282+0000","last_change":"2024-09-11T08:43:19.626282+0000","last_active":"2024-09-11T08:43:19.626282+0000","last_peered":"2024-09-11T08:43:19.626282+0000","last_clean":"2024-09-11T08:43:19.626282+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:19.626282+0000","last_undegraded":"2024-09-11T08:43:19.626282+0000","last_fullsized":"2024-09-11T08:43:19.626282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:10.938383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049065999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.396159+0000","last_change":"2024-09-11T08:43:19.396159+0000","last_active":"2024-09-11T08:43:19.396159+0000","last_peered":"2024-09-11T08:43:19.396159+0000","last_clean":"2024-09-11T08:43:19.396159+0000","last_became_active":"2024-09-11T08:43:17.384040+0000","last_became_peered":"2024-09-11T08:43:17.384040+0000","last_unstale":"2024-09-11T08:43:19.396159+0000","last_undegraded":"2024-09-11T08:43:19.396159+0000","last_fullsized":"2024-09-11T08:43:19.396159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:18:33.305374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049366500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.395345+0000","last_change":"2024-09-11T08:43:19.394642+0000","last_active":"2024-09-11T08:43:19.395345+0000","last_peered":"2024-09-11T08:43:19.395345+0000","last_clean":"2024-09-11T08:43:19.395345+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:19.395345+0000","last_undegraded":"2024-09-11T08:43:19.395345+0000","last_fullsized":"2024-09-11T08:43:19.395345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:24.191731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010925100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.469399+0000","last_change":"2024-09-11T08:43:19.469399+0000","last_active":"2024-09-11T08:43:19.469399+0000","last_peered":"2024-09-11T08:43:19.469399+0000","last_clean":"2024-09-11T08:43:19.469399+0000","last_became_active":"2024-09-11T08:43:17.383480+0000","last_became_peered":"2024-09-11T08:43:17.383480+0000","last_unstale":"2024-09-11T08:43:19.469399+0000","last_undegraded":"2024-09-11T08:43:19.469399+0000","last_fullsized":"2024-09-11T08:43:19.469399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:43.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040896800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.626257+0000","last_change":"2024-09-11T08:43:19.626257+0000","last_active":"2024-09-11T08:43:19.626257+0000","last_peered":"2024-09-11T08:43:19.626257+0000","last_clean":"2024-09-11T08:43:19.626257+0000","last_became_active":"2024-09-11T08:43:17.383933+0000","last_became_peered":"2024-09-11T08:43:17.383933+0000","last_unstale":"2024-09-11T08:43:19.626257+0000","last_undegraded":"2024-09-11T08:43:19.626257+0000","last_fullsized":"2024-09-11T08:43:19.626257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:58.667236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048316400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.394034+0000","last_change":"2024-09-11T08:43:19.394034+0000","last_active":"2024-09-11T08:43:19.394034+0000","last_peered":"2024-09-11T08:43:19.394034+0000","last_clean":"2024-09-11T08:43:19.394034+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:43:19.394034+0000","last_undegraded":"2024-09-11T08:43:19.394034+0000","last_fullsized":"2024-09-11T08:43:19.394034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:54.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719799999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T08:43:19.625706+0000","last_change":"2024-09-11T08:43:10.406953+0000","last_active":"2024-09-11T08:43:19.625706+0000","last_peered":"2024-09-11T08:43:19.625706+0000","last_clean":"2024-09-11T08:43:19.625706+0000","last_became_active":"2024-09-11T08:43:10.405743+0000","last_became_peered":"2024-09-11T08:43:10.405743+0000","last_unstale":"2024-09-11T08:43:19.625706+0000","last_undegraded":"2024-09-11T08:43:19.625706+0000","last_fullsized":"2024-09-11T08:43:19.625706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:41.277131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":655631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":802816,"data_stored":661463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35552,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336288,"statfs":{"total":96636764160,"available":96600358912,"internally_reserved":0,"allocated":208896,"data_stored":71095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:24.910 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T08:43:24.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-11T08:43:25.203 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 21 2024-09-11T08:43:25.203 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 75755954-09e8-48d3-b9c1-83899efaff39 2024-09-11T08:43:25.203 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-11T08:42:57.181838+0000 2024-09-11T08:43:25.203 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-09-11T08:43:19.384410+0000 2024-09-11T08:43:25.203 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 5 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client jewel 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout: removed_snaps_queue [2~1] 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 8 2024-09-11T08:43:25.204 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6808/2540486849,v1:172.21.15.45:6809/2540486849] [v2:172.21.15.45:6810/2540486849,v1:172.21.15.45:6811/2540486849] exists,up ac6b86cf-4ee8-49eb-a680-a350b1c9879c 2024-09-11T08:43:25.205 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6824/1040844445,v1:172.21.15.45:6825/1040844445] [v2:172.21.15.45:6826/1040844445,v1:172.21.15.45:6827/1040844445] exists,up e27d776b-f77b-4111-a536-5f5d8030c696 2024-09-11T08:43:25.205 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6816/1231172466,v1:172.21.15.45:6817/1231172466] [v2:172.21.15.45:6818/1231172466,v1:172.21.15.45:6819/1231172466] exists,up 6b8e44f6-395b-447d-a5d3-c19a0a4be61b 2024-09-11T08:43:25.205 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6800/226138367,v1:172.21.15.45:6801/226138367] [v2:172.21.15.45:6802/226138367,v1:172.21.15.45:6803/226138367] exists,up 1d1e1983-4c89-4cb4-a6b4-af244f941a7a 2024-09-11T08:43:25.205 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/374760845,v1:172.21.15.86:6801/374760845] [v2:172.21.15.86:6802/374760845,v1:172.21.15.86:6803/374760845] exists,up fd9239ee-3a74-4399-af2f-1e9a63a2f642 2024-09-11T08:43:25.205 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6824/498386186,v1:172.21.15.86:6825/498386186] [v2:172.21.15.86:6826/498386186,v1:172.21.15.86:6827/498386186] exists,up cfff2555-5ad4-4629-a725-afadf9b6a96e 2024-09-11T08:43:25.205 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6808/1261869816,v1:172.21.15.86:6809/1261869816] [v2:172.21.15.86:6810/1261869816,v1:172.21.15.86:6811/1261869816] exists,up 4588b13e-af90-48a7-a10d-f7a185a902e2 2024-09-11T08:43:25.205 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6816/2975777507,v1:172.21.15.86:6817/2975777507] [v2:172.21.15.86:6818/2975777507,v1:172.21.15.86:6819/2975777507] exists,up 12da1d21-9c59-4462-9d5a-61801dab3e4d 2024-09-11T08:43:25.216 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6808/2540486849,v1:172.21.15.45:6809/2540486849] [v2:172.21.15.45:6810/2540486849,v1:172.21.15.45:6811/2540486849] exists,up ac6b86cf-4ee8-49eb-a680-a350b1c9879c', 'osd.1 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6824/1040844445,v1:172.21.15.45:6825/1040844445] [v2:172.21.15.45:6826/1040844445,v1:172.21.15.45:6827/1040844445] exists,up e27d776b-f77b-4111-a536-5f5d8030c696', 'osd.2 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6816/1231172466,v1:172.21.15.45:6817/1231172466] [v2:172.21.15.45:6818/1231172466,v1:172.21.15.45:6819/1231172466] exists,up 6b8e44f6-395b-447d-a5d3-c19a0a4be61b', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6800/226138367,v1:172.21.15.45:6801/226138367] [v2:172.21.15.45:6802/226138367,v1:172.21.15.45:6803/226138367] exists,up 1d1e1983-4c89-4cb4-a6b4-af244f941a7a', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/374760845,v1:172.21.15.86:6801/374760845] [v2:172.21.15.86:6802/374760845,v1:172.21.15.86:6803/374760845] exists,up fd9239ee-3a74-4399-af2f-1e9a63a2f642', 'osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6824/498386186,v1:172.21.15.86:6825/498386186] [v2:172.21.15.86:6826/498386186,v1:172.21.15.86:6827/498386186] exists,up cfff2555-5ad4-4629-a725-afadf9b6a96e', 'osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6808/1261869816,v1:172.21.15.86:6809/1261869816] [v2:172.21.15.86:6810/1261869816,v1:172.21.15.86:6811/1261869816] exists,up 4588b13e-af90-48a7-a10d-f7a185a902e2', 'osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6816/2975777507,v1:172.21.15.86:6817/2975777507] [v2:172.21.15.86:6818/2975777507,v1:172.21.15.86:6819/2975777507] exists,up 12da1d21-9c59-4462-9d5a-61801dab3e4d'] 2024-09-11T08:43:25.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-11T08:43:25.318 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mon.a", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster": "ceph", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "admin_socket_mode": "", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_client_required": "cephx, none", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_cluster_required": "cephx", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_debug": "true", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_service_required": "cephx", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-11T08:43:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_supported": "cephx", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_aio": "true", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_aio_poll_ms": "250", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_aio_reap_max": "16", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_async_discard": "false", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_block_size": "4096", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_debug_aio": "false", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_debug_inflight_ios": "false", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_enable_discard": "false", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_flock_retry": "3", 2024-09-11T08:43:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_inject_crash": "0", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_ioring": "false", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_ioring_hipri": "false", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_type": "", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_alloc_size": "1048576", 2024-09-11T08:43:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_allocator": "hybrid", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_buffered_io": "true", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_check_for_zeros": "false", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_compact_log_sync": "false", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_max_log_runway": "4194304", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_max_prefetch": "1048576", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_min_flush_size": "524288", 2024-09-11T08:43:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_min_log_runway": "1048576", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_replay_recovery": "false", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_sync_write": "false", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_allocation_from_file": "true", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_allocator": "hybrid", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-11T08:43:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bdev_label_multi": "true", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_create": "true", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_db_create": "false", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_db_path": "", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_db_size": "0", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_path": "", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_preallocate_file": "false", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_size": "107374182400", 2024-09-11T08:43:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_wal_create": "false", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_wal_path": "", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_wal_size": "100663296", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bluefs": "true", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-11T08:43:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_autotune": "true", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_size": "0", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_type": "2q", 2024-09-11T08:43:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_clone_cow": "true", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_mode": "none", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_csum_type": "crc32c", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-11T08:43:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_freelist": "false", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_small_allocations": "0", 2024-09-11T08:43:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_default_buffered_read": "true", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_default_buffered_write": "false", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fail_eio": "false", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-11T08:43:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_mount": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_umount": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-11T08:43:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_ignore_data_csum": "false", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_log_op_age": "5.000000", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_alloc_size": "0", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_blob_size": "0", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-11T08:43:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_deferred_txc": "32", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_min_alloc_size": "0", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_nid_prealloc": "1024", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-11T08:43:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_retry_disk_reads": "3", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_rocksdb_cf": "true", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_spdk_mem": "512", 2024-09-11T08:43:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_tracing": "false", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-11T08:43:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_zero_block_detection": "false", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-11T08:43:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_cluster_require_signatures": "false", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_cluster_require_version": "2", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_require_signatures": "false", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_require_version": "2", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_service_require_signatures": "false", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_service_require_version": "2", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_sign_messages": "true", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "chdir": "", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "client_acl_type": "", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "client_asio_thread_count": "2", 2024-09-11T08:43:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "client_cache_mid": "0.750000", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_cache_size": "16384", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_caps_release_delay": "5", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_check_pool_perm": "true", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_debug_force_sync_read": "false", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_debug_getattr_caps": "false", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_debug_inject_features": "", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_debug_inject_tick_delay": "0", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_die_on_failed_remount": "false", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_dirsize_rbytes": "true", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_force_lazyio": "false", 2024-09-11T08:43:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "client_fs": "", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_inject_release_failure": "false", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_max_inline_size": "4096", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mds_namespace": "", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_metadata": "", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mount_gid": "-1", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mount_timeout": "300", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mount_uid": "-1", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mountpoint": "/", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_notify_timeout": "10", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc": "true", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_max_dirty": "104857600", 2024-09-11T08:43:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_max_objects": "1000", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_size": "209715200", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_target_dirty": "8388608", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_permissions": "true", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_quota": "true", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_quota_df": "true", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_readahead_max_bytes": "0", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_readahead_max_periods": "4", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_readahead_min": "131072", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_reconnect_stale": "false", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_shutdown_timeout": "30", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_snapdir": ".snap", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_tick_interval": "1", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_trace": "", 2024-09-11T08:43:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_try_dentry_invalidate": "false", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "client_use_faked_inos": "false", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "client_use_random_mds": "false", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_graylog": "false", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_graylog_port": "12201", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_monitors": "default=true", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_syslog": "false", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_syslog_level": "info", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "-", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_network": "", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_network_interface": "", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "compressor_zlib_isal": "false", 2024-09-11T08:43:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "compressor_zlib_level": "5", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "compressor_zlib_winsize": "-15", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "compressor_zstd_level": "1", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_alien_op_num_threads": "6", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_osd_stat_interval": "0", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_seastar_cpu_cores": "", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_seastar_num_threads": "0", 2024-09-11T08:43:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location_hook": "", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location_hook_timeout": "10", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "daemonize": "false", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "daos_pool": "tank", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_alienstore": "0/5", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_allow_any_pool_priority": "false", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_asok": "1/5", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_asok_assert_abort": "false", 2024-09-11T08:43:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_asserts_on_shutdown": "true", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_auth": "20/20", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_bdev": "1/3", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_bluefs": "1/5", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_bluestore": "1/5", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_buffer": "0/1", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_ceph_exporter": "1/5", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_cephfs_mirror": "0/5", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_cephsqlite": "0/5", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_client": "0/5", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_compressor": "1/5", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_context": "0/1", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_crimson_interrupt": "0/5", 2024-09-11T08:43:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_crush": "1/1", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_crypto": "1/5", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_cyanstore": "0/5", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_deliberately_leak_memory": "false", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_disable_randomized_ping": "false", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_dpdk": "1/5", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_eventtrace": "1/5", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_filer": "0/1", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_filestore": "1/3", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_finisher": "1/1", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_fuse": "1/5", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_heartbeat_testing_span": "0", 2024-09-11T08:43:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_heartbeatmap": "1/5", 2024-09-11T08:43:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-11T08:43:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_javaclient": "1/5", 2024-09-11T08:43:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_journal": "1/3", 2024-09-11T08:43:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_journaler": "0/5", 2024-09-11T08:43:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_kstore": "1/5", 2024-09-11T08:43:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_lockdep": "0/1", 2024-09-11T08:43:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mclock": "1/5", 2024-09-11T08:43:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds": "1/5", 2024-09-11T08:43:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_balancer": "1/5", 2024-09-11T08:43:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_locker": "1/5", 2024-09-11T08:43:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_log": "1/5", 2024-09-11T08:43:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_log_expire": "1/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_migrator": "1/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_quiesce": "3/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_memstore": "1/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mgr": "2/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mgrc": "1/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mon": "20/20", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_monc": "0/10", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_ms": "1/1", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_none": "0/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_objclass": "0/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_objectcacher": "0/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_objecter": "0/1", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_optracker": "0/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_osd": "1/5", 2024-09-11T08:43:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_paxos": "20/20", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_perfcounter": "1/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_prioritycache": "1/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rados": "0/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rbd": "0/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rbd_mirror": "0/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rbd_pwl": "0/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rbd_replay": "0/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_refs": "0/0", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_reserver": "1/1", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw": "1/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_access": "1/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_datacache": "1/5", 2024-09-11T08:43:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_dbstore": "1/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_flight": "1/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_sync": "1/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rocksdb": "4/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore": "0/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_backref": "0/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_cache": "0/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_cleaner": "0/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_device": "0/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_epm": "0/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-11T08:43:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_journal": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_lba": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_odata": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_omap": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_onode": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_t": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_tm": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_striper": "0/1", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_test": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_throttle": "1/1", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_timer": "0/1", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_tp": "0/5", 2024-09-11T08:43:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_trace": "1/5", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "device_failure_prediction_mode": "none", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "err_to_graylog": "false", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "err_to_journald": "false", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "err_to_stderr": "true", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "err_to_syslog": "false", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event_tracing": "false", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_addr": "0.0.0.0", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_http_port": "9926", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_prio_limit": "5", 2024-09-11T08:43:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_sort_metrics": "true", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_stats_period": "5", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "fake_statfs_for_testing": "0", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "fatal_signal_handlers": "true", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "filer_max_purge_ops": "10", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "filer_max_truncate_ops": "128", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_apply_finisher_threads": "1", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_blackhole": "false", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_btrfs_clone_range": "true", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_btrfs_snap": "true", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_caller_concurrency": "10", 2024-09-11T08:43:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_collect_device_partition_information": "true", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_commit_timeout": "600.000000", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_debug_inject_read_err": "false", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_debug_omap_check": "false", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_debug_verify_split": "false", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_dump_file": "", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fadvise": "true", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fail_eio": "true", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fd_cache_shards": "16", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fd_cache_size": "128", 2024-09-11T08:43:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fiemap": "false", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fiemap_threshold": "4096", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_inject_stall": "0", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_journal_parallel": "false", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_journal_trailing": "false", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_journal_writeahead": "false", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_kill_at": "0", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-11T08:43:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattrs": "0", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_xattr_value_size": "0", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_merge_threshold": "-10", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_odsync_write": "false", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_omap_backend": "rocksdb", 2024-09-11T08:43:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_omap_backend_path": "", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_op_thread_timeout": "60", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_op_threads": "2", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_punch_hole": "false", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-11T08:43:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_ops": "50", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_seek_data_hole": "false", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_sloppy_crc": "false", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_splice": "false", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_split_multiple": "2", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_split_rand_factor": "20", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_update_to": "1000", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-11T08:43:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_enable": "true", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_xfs_extsize": "false", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_zfs_snap": "false", 2024-09-11T08:43:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "fio_dir": "/tmp/fio", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "75755954-09e8-48d3-b9c1-83899efaff39", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_allow_other": "true", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_atomic_o_trunc": "true", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_big_writes": "true", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_debug": "false", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_default_permissions": "false", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_disable_pagecache": "false", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_max_write": "0", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_multithreaded": "true", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_require_active_mds": "true", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_set_user_groups": "true", 2024-09-11T08:43:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_splice_move": "true", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_splice_read": "true", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_splice_write": "true", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_use_invalidate_cb": "true", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "gss_target_name": "ceph", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_file": "", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_inject_failure": "0", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_interval": "5", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "host": "smithi045", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-11T08:43:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "inject_early_sigterm": "false", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "jaeger_agent_port": "6799", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "jaeger_tracing_enable": "false", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_aio": "true", 2024-09-11T08:43:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_align_min_size": "65536", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_block_align": "true", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_block_size": "4096", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_dio": "true", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_discard": "false", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_force_aio": "false", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_ignore_corruption": "false", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_max_write_bytes": "10485760", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_max_write_entries": "100", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_replay_from": "0", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-11T08:43:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_write_header_frequency": "0", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_zero_on_create": "false", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "journaler_prefetch_periods": "10", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "journaler_prezero_periods": "5", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "journaler_write_head_interval": "15", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "keyfile": "", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_backend": "rocksdb", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_default_stripe_size": "65536", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_fsck_on_mount": "false", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-11T08:43:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_max_bytes": "67108864", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_max_ops": "512", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_nid_prealloc": "1024", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_onode_map_size": "1024", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_sync_submit_transaction": "false", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_sync_transaction": "false", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "librados_thread_count": "2", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "lockdep": "false", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "lockdep_force_backtrace": "false", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "log_coarse_timestamps": "true", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-11T08:43:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "log_flush_on_exit": "false", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_graylog_host": "127.0.0.1", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_graylog_port": "12201", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_max_new": "1000", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_max_recent": "10000", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_stderr_prefix": "", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_stop_at_utilization": "0.970000", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": "true", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_graylog": "false", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_journald": "false", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_stderr": "false", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_syslog": "false", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max_rotating_auth_attempts": "10", 2024-09-11T08:43:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_action_on_write_error": "1", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_alternate_name_max": "8192", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_asio_thread_count": "2", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_export_pin": "true", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_fragment_dirs": "true", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_fragment_interval": "5", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_interval": "10", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_max": "-1", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_max_until": "-1", 2024-09-11T08:43:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_merge_size": "50", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_midchunk": "0.300000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_min_start": "0.200000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_minchunk": "0.001000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_mode": "0", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_need_max": "1.200000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_need_min": "0.800000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_overload_epochs": "2", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_split_bits": "3", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_split_size": "10000", 2024-09-11T08:43:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_target_decay": "10.000000", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_beacon_grace": "15.000000", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_beacon_interval": "4.000000", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_mid": "0.700000", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_delay": "0", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_release_free_interval": "10", 2024-09-11T08:43:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_reservation": "0.050000", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_trim_interval": "1", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_trim_threshold": "262144", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_client_prealloc_inos": "1000", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_connect_bootstrapping": "false", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_damage_table_max_entries": "10000", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_debug_auth_pins": "false", 2024-09-11T08:43:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_debug_frag": "false", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_debug_scatterstat": "false", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_debug_subtrees": "false", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_decay_halflife": "5.000000", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_default_dir_hash": "2", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_defer_session_stale": "true", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_deny_all_reconnect": "false", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dir_keys_per_op": "16384", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dir_max_commit_size": "10", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dir_max_entries": "0", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dir_prefetch": "true", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-11T08:43:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dump_cache_on_map": "false", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_early_reply": "true", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_enable_op_tracker": "true", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_enforce_unique_name": "true", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_export_ephemeral_random": "true", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-11T08:43:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_health_summarize_threshold": "10", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_health_dummy": "false", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_migrator_session_race": "false", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-11T08:43:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_join_fs": "", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_journal_format": "1", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_create_at": "0", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_export_at": "0", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_import_at": "0", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_journal_at": "0", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_journal_expire_at": "0", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_journal_replay_at": "0", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_link_at": "0", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_mdstable_at": "0", 2024-09-11T08:43:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_openc_at": "0", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_rename_at": "0", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_shutdown_at": "0", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_event_large_threshold": "524288", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_events_per_segment": "1024", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_max_events": "-1", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_max_segments": "128", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_pause": "false", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_segment_size": "0", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_trim_threshold": "128", 2024-09-11T08:43:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_warn_factor": "2.000000", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_caps_per_client": "1048576", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_completed_flushes": "100000", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_completed_requests": "100000", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_export_size": "20971520", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_file_recover": "32", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_purge_files": "64", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_purge_ops": "8192", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_snaps_per_dir": "100", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_metrics_update_interval": "2", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_min_caps_per_client": "100", 2024-09-11T08:43:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_min_caps_working_set": "10000", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_numa_node": "-1", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_complaint_time": "30.000000", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_history_duration": "600", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_history_size": "20", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_history_slow_op_size": "20", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_log_threshold": "5", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_ping_grace": "15", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_ping_interval": "5", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-11T08:43:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_max_caps": "30000", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_warning_threshold": "262144", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_replay_interval": "1.000000", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_root_ino_gid": "0", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_root_ino_uid": "0", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-11T08:43:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_shutdown_check": "0", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_skip_ino": "0", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_snap_max_uid": "4294967294", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_snap_min_uid": "0", 2024-09-11T08:43:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_snap_rstat": "false", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_standby_replay_damaged": "false", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_symlink_recovery": "true", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_thrash_exports": "0", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_thrash_fragments": "0", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_tick_interval": "5.000000", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_valgrind_exit": "false", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_verify_backtrace": "1", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_verify_scatter": "false", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_wipe_sessions": "false", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool_debug": "false", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "memstore_device_bytes": "1073741824", 2024-09-11T08:43:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "memstore_page_set": "false", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "memstore_page_size": "65536", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_client_bytes": "134217728", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_client_messages": "512", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_disabled_modules": "", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_enable_op_tracker": "true", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_max_pg_num_change": "128", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_mds_bytes": "134217728", 2024-09-11T08:43:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_mds_messages": "128", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_mon_bytes": "134217728", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_mon_messages": "128", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_history_duration": "600", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_history_size": "20", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_log_threshold": "5", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_osd_bytes": "536870912", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_osd_messages": "8192", 2024-09-11T08:43:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_pool": "true", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_standby_modules": "true", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_stats_period": "5", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_stats_threshold": "5", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_tick_period": "2", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_allow_pool_delete": "true", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_allow_pool_size_one": "true", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_auth_validate_all_caps": "true", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-11T08:43:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_bytes": "104857600", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_directed_command_retry": "5", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_parallel": "3", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_log_interval": "1.000000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_ping_interval": "10.000000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_target_rank": "-1", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-11T08:43:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_file_level": "debug", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_file": "true", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_journald": "false", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_compact_on_bootstrap": "false", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_compact_on_start": "false", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_compact_on_trim": "true", 2024-09-11T08:43:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cpu_threads": "4", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_crush_min_required_version": "hammer", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_daemon_bytes": "419430400", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_data_avail_crit": "5", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_data_avail_warn": "5", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_data_size_warn": "16106127360", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_dump_json": "false", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-11T08:43:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_dump_transactions": "false", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_extra_checks": "false", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_no_require_reef": "false", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_no_require_squid": "false", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_down_mkfs_grace": "120", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_down_uptime_grace": "60", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_election_default_strategy": "1", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_election_timeout": "5.000000", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-11T08:43:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_elector_ping_divisor": "2", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_enable_op_tracker": "true", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_fake_pool_delete": "false", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_force_quorum_join": "false", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_globalid_prealloc": "10000", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_detail_to_clog": "false", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_log_update_period": "5", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_max_detail": "50", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_to_clog": "true", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_to_clog_interval": "600", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-11T08:43:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_host": "172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790],172.21.15.86", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_host_override": "", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_initial_members": "", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_lease": "5.000000", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_log_full_interval": "50", 2024-09-11T08:43:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_log_max": "10000", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_log_max_summary": "50", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_log_epochs": "500", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_osd": "10000", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_pg_per_osd": "10000", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_pool_pg_num": "65536", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mds_force_trim_to": "0", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mds_skip_sanity": "false", 2024-09-11T08:43:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_memory_autotune": "true", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_memory_target": "2147483648", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_beacon_grace": "30", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_digest_period": "5", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_inactive_grace": "60", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_min_osdmap_epochs": "500", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_complaint_time": "30", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_history_duration": "600", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_history_size": "20", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_history_slow_op_size": "20", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-11T08:43:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_log_threshold": "5", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_auto_mark_in": "false", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_cache_size": "500", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_down_out_interval": "600", 2024-09-11T08:43:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_force_trim_to": "0", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_min_down_reporters": "2", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-11T08:43:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_report_timeout": "900", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-11T08:43:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_stuck_threshold": "60", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_probe_timeout": "2.000000", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_reweight_max_change": "0.050000", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_reweight_max_osds": "4", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-11T08:43:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_interval": "86400", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_max_keys": "100", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_timeout": "300", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_session_timeout": "300", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_smart_report_timeout": "5", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stat_smooth_intervals": "6", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stretch_pool_min_size": "2", 2024-09-11T08:43:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stretch_pool_size": "4", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_debug": "false", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_provider_kill_at": "0", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_requester_kill_at": "0", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_timeout": "60.000000", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_target_pg_per_osd": "100", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_tick_interval": "5", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_timecheck_interval": "300.000000", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_use_min_delay_socket": "false", 2024-09-11T08:43:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_older_version_delay": "604800", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_misplaced": "false", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_older_version": "true", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-11T08:43:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": "", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-11T08:43:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_tracing_enabled": "false", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_op_threads": "3", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_cm": "false", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_device_name": "", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_dscp": "96", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_local_gid": "", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_port_num": "1", 2024-09-11T08:43:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_sl": "3", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_support_srq": "true", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_type": "ib", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_reap_threshold": "5", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_before_connect": "false", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_ipv4": "true", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_ipv6": "false", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_msgr1": "true", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_msgr2": "true", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_port_max": "7568", 2024-09-11T08:43:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_port_min": "6800", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_retry_count": "3", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_retry_delay": "5", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_client": "false", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_mds": "false", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_mgr": "false", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_mon": "false", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_osd": "false", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_client_mode": "crc secure", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_cluster_mode": "crc secure", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_cluster_type": "", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_compress_secure": "false", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_connection_idle_timeout": "900", 2024-09-11T08:43:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_connection_ready_timeout": "10", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_crc_data": "true", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_crc_header": "true", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_bad_msg": "false", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_bug": "true", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_old_message": "true", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_skipped_message": "false", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_coremask": "0xF", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_enable_tso": "true", 2024-09-11T08:43:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_hugepages": "", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_lro": "true", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_memory_channel": "4", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_pmd": "", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_port_id": "0", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dump_on_send": "false", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_initial_backoff": "0.200000", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_delay_max": "1.000000", 2024-09-11T08:43:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_delay_type": "", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_network_congestion": "0", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_socket_failures": "5000", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_learn_addr_from_peer": "true", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_max_accept_failures": "4", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_max_backoff": "15.000000", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_mon_client_mode": "secure crc", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_mon_service_mode": "secure crc", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_osd_compress_min_size": "1024", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_osd_compress_mode": "none", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-11T08:43:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_pq_min_cost": "65536", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_public_type": "", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_service_mode": "crc secure", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_tcp_listen_backlog": "512", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_tcp_nodelay": "true", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_tcp_rcvbuf": "0", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_type": "async+posix", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "no_config_file": "false", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_completion_locks_per_session": "32", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_inflight_ops": "1024", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-11T08:43:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_tick_interval": "5.000000", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_timeout": "10.000000", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "objectstore_blackhole": "false", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "openssl_engine_opts": "", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_delay_time": "5.000000", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_hist_halflife": "1000", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_max_low_ops": "2", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_max_ops": "4", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_slop": "0.020000", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-11T08:43:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_asio_thread_count": "2", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_async_recovery_min_cost": "100", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backfill_scan_max": "512", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backfill_scan_min": "64", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backoff_on_degraded": "false", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backoff_on_peering": "false", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backoff_on_unfound": "true", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_beacon_report_interval": "300", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bench_duration": "30", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bench_max_block_size": "67108864", 2024-09-11T08:43:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_blkin_trace_all": "false", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_check_for_log_corruption": "false", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_class_update_on_start": "true", 2024-09-11T08:43:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_client_message_cap": "256", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_client_message_size_cap": "524288000", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_client_op_priority": "63", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_client_watch_timeout": "30", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_command_max_records": "256", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_compact_on_start": "false", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_crush_update_on_start": "true", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_crush_update_weight_set": "true", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-11T08:43:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_feed_pullee": "-1", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_misdirected_ops": "false", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_no_acting_change": "false", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_no_purge_strays": "false", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_op_order": "false", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-11T08:43:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_shutdown": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_trim_objects": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_verify_snaps": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-11T08:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_keys": "1024", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_stride": "524288", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_default_notify_timeout": "30", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_delete_sleep": "0.000000", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_discard_disconnected_ops": "true", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_enable_op_tracker": "true", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-11T08:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_info": "true", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_shutdown": "true", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_function_tracing": "false", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_grace": "20", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_interval": "6", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-11T08:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_min_peers": "10", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_min_size": "2000", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_stale": "600", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_hit_set_max_size": "100000", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_hit_set_min_size": "1000", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_journal_size": "5120", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_kill_backfill_at": "0", 2024-09-11T08:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_cache_size": "50", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_dedup": "true", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_message_max": "40", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_share_max_epochs": "40", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_attr_name_len": "100", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_attr_size": "0", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_backfills": "1", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_markdown_count": "5", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_markdown_period": "600", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_object_name_len": "2048", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_object_namespace_len": "256", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_object_size": "134217728", 2024-09-11T08:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_pg_blocked_by": "16", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_pg_log_entries": "10000", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_pgls": "1024", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_push_cost": "8388608", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_push_objects": "10", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_scrubs": "3", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_trimming_pgs": "2", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_write_op_reply_len": "64", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_write_size": "90", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-11T08:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_profile": "balanced", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-11T08:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_base": "805306368", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_cache_min": "134217728", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-11T08:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_target": "4294967296", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_target_autotune": "false", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_min_pg_log_entries": "250", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_min_recovery_priority": "0", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_report_interval": "5", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_num_cache_shards": "32", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_num_op_tracker_shard": "32", 2024-09-11T08:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_numa_auto_affinity": "true", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_numa_node": "-1", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_numa_prefer_iface": "true", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objecter_finishers": "1", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objectstore": "bluestore", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objectstore_fuse": "false", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objectstore_tracing": "false", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_complaint_time": "30.000000", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_history_duration": "600", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_history_size": "20", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_history_slow_op_size": "20", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_log_threshold": "5", 2024-09-11T08:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_shards": "0", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_shards_hdd": "5", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_shards_ssd": "8", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_pq_min_cost": "65536", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_queue_cut_off": "high", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_thread_timeout": "15", 2024-09-11T08:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_open_classes_on_start": "true", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_os_flags": "0", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_peering_op_priority": "255", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_delete_cost": "1048576", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_delete_priority": "5", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_log_trim_max": "10000", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_log_trim_min": "100", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-11T08:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_crimson": "false", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-11T08:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flags": "0", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_min_size": "0", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_pg_num": "32", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_pgp_num": "0", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_read_ratio": "70", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_size": "2", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_type": "replicated", 2024-09-11T08:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_push_per_object_cost": "1000", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recover_clone_overlap": "true", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_cost": "20971520", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_active": "0", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_single_start": "1", 2024-09-11T08:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_op_priority": "3", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_priority": "5", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_sleep": "0.000000", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_repair_during_recovery": "false", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_requested_scrub_priority": "120", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-11T08:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_auto_repair": "false", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_begin_hour": "0", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_begin_week_day": "0", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_chunk_max": "25", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_chunk_min": "5", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_cost": "52428800", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_during_recovery": "false", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_end_hour": "0", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_end_week_day": "0", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_event_cost": "4096", 2024-09-11T08:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_invalid_stats": "true", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_max_preemptions": "5", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_priority": "5", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_sleep": "0.000000", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-11T08:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_skip_data_digest": "false", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_smart_report_timeout": "5", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_cost": "1048576", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_priority": "5", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_target_transaction_size": "30", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-11T08:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tracing": "false", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_use_stale_snap": "false", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osdc_blkin_trace_all": "false", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_kill_at": "0", 2024-09-11T08:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_max_join_drift": "10", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_min": "500", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_min_wait": "0.050000", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_propose_interval": "1.000000", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_service_trim_max": "500", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_service_trim_min": "250", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_stash_full_interval": "25", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_trim_max": "500", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_trim_min": "250", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "perf": "true", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-11T08:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "-", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrv": "", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_bind_addr": "-", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_network": "", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "public_network_interface": "", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "qat_compressor_busy_polling": "false", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "qat_compressor_enabled": "false", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "qat_compressor_session_max_number": "256", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rados_mon_op_timeout": "0", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rados_osd_op_timeout": "0", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rados_tracing": "false", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_atime_update_interval": "60", 2024-09-11T08:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_balance_parent_reads": "false", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_balance_snap_reads": "false", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_blkin_trace_all": "false", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache": "true", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-11T08:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_policy": "writearound", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_size": "33554432", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_clone_copy_on_read": "false", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_compression_hint": "none", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_concurrent_management_ops": "10", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_clone_format": "auto", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_data_pool": "", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_features": "61", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_format": "2", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_map_options": "", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_order": "22", 2024-09-11T08:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_pool": "rbd", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_stripe_count": "0", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_stripe_unit": "0", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_enable_alloc_hint": "true", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_io_scheduler": "simple", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-11T08:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_order": "24", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_pool": "", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_splay_width": "4", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_localize_parent_reads": "false", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_localize_snap_reads": "false", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-11T08:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-11T08:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mtime_update_interval": "60", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_non_blocking_aio": "true", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_op_thread_timeout": "60", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_op_threads": "1", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_parent_cache_enabled": "false", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_plugins": "", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_bps_burst": "0", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-11T08:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_bps_limit": "0", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_exclude_ops": "0", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_iops_burst": "0", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_iops_limit": "0", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-11T08:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_read_from_replica_policy": "default", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_skip_partial_discard": "true", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-11T08:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_tracing": "false", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_validate_names": "true", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_validate_pool": "true", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "restapi_base_url": "", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "restapi_log_level": "", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_acl_grants_max_num": "100", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_admin_entry": "admin", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_backend_store": "rados", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_barbican_url": "", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_beast_enable_async": "true", 2024-09-11T08:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_counters_cache": "false", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cache_enabled": "true", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cache_expiry_interval": "900", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cache_lru_size": "25000", 2024-09-11T08:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_config_store": "rados", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_content_length_compat": "false", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_copy_obj_progress": "true", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cors_rules_max_num": "100", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cross_domain_policy": "", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_addr": "", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_password": "", 2024-09-11T08:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_username": "", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_require_ssl": "true", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-11T08:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_addr": "", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_auth": "token", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_namespace": "", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_prefix": "", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-11T08:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_token_file": "", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_buffersize": "524288", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_low_speed_time": "300", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-11T08:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_port": "6379", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_log_changes_size": "1000", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_log_num_shards": "128", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_log_window": "30", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-11T08:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-11T08:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dns_name": "", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dns_s3website_name": "", 2024-09-11T08:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dynamic_resharding": "true", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_gc_threads": "true", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_lc_threads": "true", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_ops_log": "false", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_quota_threads": "true", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_static_website": "false", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_usage_log": "false", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enforce_swift_acls": "true", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_exit_timeout_secs": "120", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_expose_bucket": "false", 2024-09-11T08:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_extended_http_attrs": "", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_filter": "none", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_frontends": "beast port=7480", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_deferred": "50", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_objs": "32", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_processor_period": "3600", 2024-09-11T08:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_init_timeout": "300", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_kafka_connection_idle": "300", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-11T08:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_domain": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_password": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_password_path": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_project": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_tenant": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_token": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_token_path": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_user": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_api_version": "2", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_domain": "", 2024-09-11T08:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_password": "", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_project": "", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_user": "", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_url": "", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-11T08:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_debug_interval": "-1", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_lock_max_time": "90", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_max_objs": "32", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_max_rules": "1000", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_max_worker": "3", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_thread_delay": "0", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_dnattr": "uid", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_searchfilter": "", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_uri": "", 2024-09-11T08:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_log_http_headers": "", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_log_object_name_utc": "false", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_attr_name_len": "0", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_attr_size": "0", 2024-09-11T08:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_chunk_size": "4194304", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_listing_results": "1000", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_notify_retries": "10", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_put_param_size": "1048576", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_put_size": "5368709120", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_slo_entries": "1000", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_md_log_max_shards": "64", 2024-09-11T08:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_mp_lock_max_time": "600", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-11T08:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_max_gc": "300", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_num_async_rados_threads": "32", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_num_control_oids": "8", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_numa_node": "-1", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-11T08:43:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_objexp_chunk_size": "100", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_objexp_gc_interval": "600", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_op_thread_timeout": "600", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_op_tracing": "false", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_opa_token": "", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_opa_url": "", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_opa_verify_ssl": "true", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-11T08:43:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ops_log_rados": "false", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ops_log_socket_path": "", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_period_push_interval": "2.000000", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_cache_lanes": "3", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-11T08:43:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_cache_partitions": "3", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_print_continue": "true", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_rados_tracing": "false", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_realm": "", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_realm_id": "", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_region": "", 2024-09-11T08:43:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_request_uri": "", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_batch_size": "64", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_max_aio": "128", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_num_logs": "16", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_thread_interval": "600", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_resolve_cname": "false", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_run_sync_thread": "true", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-11T08:43:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_scheduler_type": "throttler", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_script_uri": "", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_service_provider_name": "", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_client_id": "", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_client_secret": "", 2024-09-11T08:43:25.468 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_entry": "sts", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_key": "sts", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_min_session_duration": "900", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_token_introspection_url": "", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_account_in_url": "false", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_auth_entry": "auth", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_auth_url": "", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_custom_header": "", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_need_stats": "true", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_tenant_name": "", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_token_expiration": "86400", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_url": "", 2024-09-11T08:43:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_url_prefix": "swift", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_lease_period": "120", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-11T08:43:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_thread_pool_size": "512", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_comment": "", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_createby": "", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_encoding": "", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_flag": "false", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_origin": "", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-11T08:43:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_tracker": "", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_trust_forwarded_https": "false", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_usage_max_shards": "32", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_usage_max_user_shards": "1", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_use_opa_authz": "false", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_counters_cache": "false", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_max_buckets": "1000", 2024-09-11T08:43:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_policies_max_num": "100", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_unique_email": "true", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_verify_ssl": "true", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zone": "", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zone_id": "", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zonegroup": "", 2024-09-11T08:43:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zonegroup_id": "", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rgwlc_auto_session_clear": "true", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_block_size": "4096", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_size": "536870912", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-11T08:43:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_index_type": "binary_search", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_partition_filters": "false", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_perf": "false", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-11T08:43:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "run_dir": "/var/run/ceph", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_block_create": "true", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_cache_lru_size": "67108864", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_default_max_object_size": "16777216", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_device_size": "53687091200", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_full_integrity_check": "false", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_journal_batch_capacity": "16", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-11T08:43:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_main_device_type": "SSD", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_segment_size": "67108864", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "set_keepcaps": "false", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "setgroup": "", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "setuser": "", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "setuser_match_path": "", 2024-09-11T08:43:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-11T08:43:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "thp": "false", 2024-09-11T08:43:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "threadpool_default_timeout": "60", 2024-09-11T08:43:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-11T08:43:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "throttler_perf_counter": "true" 2024-09-11T08:43:25.478 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:25.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-11T08:43:25.596 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a: {} 2024-09-11T08:43:25.596 INFO:teuthology.orchestra.run.smithi045.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:43:25.605 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b: {} 2024-09-11T08:43:25.605 INFO:teuthology.orchestra.run.smithi045.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:43:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c: {} 2024-09-11T08:43:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:43:25.625 DEBUG:teuthology.orchestra.run.smithi086:> type ceph-objectstore-tool 2024-09-11T08:43:25.642 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-11T08:43:25.643 DEBUG:teuthology.orchestra.run.smithi045:> type ceph-objectstore-tool 2024-09-11T08:43:25.657 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-11T08:43:25.658 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T08:43:25.672 INFO:tasks.workunit:Pulling workunits from ref daaf78de334647fc0c800f4ac495cc9475a26bae 2024-09-11T08:43:25.672 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T08:43:25.672 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:43:25.673 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-11T08:43:25.673 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:43:25.673 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:43:25.673 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:43:25.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-11T08:43:25.674 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-11T08:43:25.674 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-11T08:43:25.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:25.674 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-11T08:43:25.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:25.675 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-11T08:43:25.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T08:43:25.699 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:25.699+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:25.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.716 INFO:teuthology.orchestra.run.smithi045.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-11T08:43:25.716 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:43:25.776 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:25.776+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:25.779 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-11T08:43:25.779 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-11T08:43:25.822 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:25.822 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:25.822 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:25.822 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:25.834 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:25.837 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:25.837 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:25.840 INFO:tasks.workunit:timeout=3h 2024-09-11T08:43:25.840 INFO:tasks.workunit:cleanup=True 2024-09-11T08:43:25.841 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout daaf78de334647fc0c800f4ac495cc9475a26bae 2024-09-11T08:43:25.856 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:25.857 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:25.877 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T08:43:25.878 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:25.877+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:25.878 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:25.878 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:25.912 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:25.945 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:25.946 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:25.977 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:25.978 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:25.978 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:25.978+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:25.986 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:25.986 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:25.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:26.002 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:26.002 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:26.037 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:26.043 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-11T08:43:26.044+0000 7f1a550d0640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-11T08:43:26.079 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:26.080+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:26.114 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:26.114 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:26.123 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:26.180 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:26.181+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:26.231 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:26.231 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:26.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:26.281 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:26.282+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:26.348 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:26.349 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:26.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:26.382 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:26.382+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:26.407 INFO:teuthology.orchestra.run.smithi045.stderr:marked out osd.1. 2024-09-11T08:43:26.413 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-11T08:43:26.475 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:26.475 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:26.483 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:26.484+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:26.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:26.584 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:26.585+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:26.633 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:26.633 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:26.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:26.685 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:26.685+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:26.785 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:26.785+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:26.789 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:26.790 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:26.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:26.886 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:26.887+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:26.946 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:26.946 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:26.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:26.987 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:26.987+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:27.089 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:27.089+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:27.103 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:27.104 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:27.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:27.189 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:27.190+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:27.219 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:27.220 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:27.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:27.290 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:27.291+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:27.376 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:27.376 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:27.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:27.392 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:27.392+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:27.493 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:27.494+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:27.501 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:27.501 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:27.510 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:27.594 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:27.594+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:27.613 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:27.613 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:27.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:27.695 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:27.696+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:27.724 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:27.724 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:27.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:27.796 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:27.797+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:27.835 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:27.835 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:27.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:27.896 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:27.897+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:27.946 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:27.946 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:27.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:27.998 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:27.999+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:28.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:28.059 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:28.059 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:28.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:28.099 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:28.099+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:28.189 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:28.200 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:28.200+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:28.206 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:28.212 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:28.212 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:28.221 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:28.223 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:28.239 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:28.256 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:28.272 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:28.289 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:28.289 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:28.300 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:28.301+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:28.306 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:28.306 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:28.364 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:28.364 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:28.364 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:28.364 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:28.364 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:28.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:28.401 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:28.402+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:28.436 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T08:43:28.476 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:28.477 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:28.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:28.477 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:28.477 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:28.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:28.502 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:28.503+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:28.588 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:28.589 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:28.597 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:28.603 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:28.604+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:28.699 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:28.700 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:28.700 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:28.700 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:28.700 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:28.704 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:28.705+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:28.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:28.805 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:28.807+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:28.813 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 19.451759436, 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.377657073999998, 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.848 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:28.848 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.377585585999999, 2024-09-11T08:43:28.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:43:28.848 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.849 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.849 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.849 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:43:28.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.849 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:43:28.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:43:28.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:43:28.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:43:28.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.377561974999999, 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:43:28.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:43:28.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.861 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.335794591999999, 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:43:28.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:43:28.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:43:28.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.867 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.335770726, 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:43:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:43:28.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.335664668, 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:43:28.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:43:28.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.183962191999999, 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:43:28.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.183933096000001, 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.887 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:28.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:43:28.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:43:28.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:43:28.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:43:28.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:43:28.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:43:28.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.183906756999999, 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:43:28.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:43:28.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:43:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.183837457999999, 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.902 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:28.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.183812505999999, 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:43:28.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:43:28.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.774115370000001, 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.915 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:43:28.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:43:28.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.774027756999999, 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.921 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:43:28.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:43:28.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.773988248999999, 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:43:28.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:43:28.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.773962044000001, 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.934 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:43:28.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.773910788999999, 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.940 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:43:28.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.77389887, 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:43:28.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.952 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.773884992999999, 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.953 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:43:28.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:43:28.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:43:28.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.959 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.773843451000001, 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:28.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:43:28.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:43:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:28.966 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:28.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:28.968 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:28.968+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:29.068 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:29.069+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:29.076 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:29.076 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:29.076 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:29.076 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:29.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:29.169 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:29.170+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:29.233 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:29.233 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:29.233 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:29.233 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:29.233 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:29.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:29.283 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:29.283+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:29.371 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:29.372+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:29.390 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.967178996000001, 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011043540000000001, 2024-09-11T08:43:29.391 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:29.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423249+0000", 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0200000000000001e-06 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423401+0000", 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015192100000000001 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423432+0000", 2024-09-11T08:43:29.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1282999999999998e-05 2024-09-11T08:43:29.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423439+0000", 2024-09-11T08:43:29.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.241e-06 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423512+0000", 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3620000000000003e-05 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423529+0000", 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6813999999999999e-05 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424295+0000", 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076553700000000005 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424352+0000", 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6918e-05 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.955750213000002, 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011035800000000001, 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434677+0000", 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434683+0000", 2024-09-11T08:43:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.212e-06 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434701+0000", 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8067000000000002e-05 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434721+0000", 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9562999999999999e-05 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434778+0000", 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7581000000000003e-05 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434800+0000", 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2204e-05 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435738+0000", 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093753900000000004 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435780+0000", 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1683000000000001e-05 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.955286854000001, 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:43:29.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.405 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.954767064999999, 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010649730000000001, 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435660+0000", 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.51e-07 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435665+0000", 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435674+0000", 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435679+0000", 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6539999999999998e-06 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435815+0000", 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013680800000000001 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435838+0000", 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2821000000000001e-05 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436692+0000", 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085374500000000005 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436724+0000", 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2246000000000003e-05 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.410 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.954222690999998, 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:43:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'15)", 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.953728820999999, 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108423, 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436698+0000", 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4499999999999996e-07 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436737+0000", 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9226e-05 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436751+0000", 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3172e-05 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436758+0000", 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689999999999999e-06 2024-09-11T08:43:29.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436816+0000", 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7506999999999997e-05 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436836+0000", 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0324000000000001e-05 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437746+0000", 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000910537 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437782+0000", 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5150000000000001e-05 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:38 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.949236773999999, 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010117800000000001, 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441190+0000", 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9999999999999996e-07 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441257+0000", 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6607999999999997e-05 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441269+0000", 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1953999999999999e-05 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441275+0000", 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9020000000000004e-06 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441315+0000", 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0506999999999997e-05 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441328+0000", 2024-09-11T08:43:29.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3165e-05 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442139+0000", 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081040300000000001 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442201+0000", 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2440999999999994e-05 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:39 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'25, mlcod=16'22)", 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.948802560000001, 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010054160000000001, 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:29.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0900000000000002e-07 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441721+0000", 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6898000000000006e-05 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441734+0000", 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.332e-05 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441744+0000", 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.465e-06 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441812+0000", 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7694999999999995e-05 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441834+0000", 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2614000000000001e-05 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442593+0000", 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075926599999999998 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442629+0000", 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5648999999999997e-05 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.430 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:53 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'31, mlcod=16'29)", 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.937427015000001, 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098036799999999991, 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453000+0000", 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0400000000000005e-07 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453012+0000", 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1911e-05 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453024+0000", 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2257e-05 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453030+0000", 2024-09-11T08:43:29.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0009999999999998e-06 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453076+0000", 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5516000000000003e-05 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453096+0000", 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0361e-05 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453934+0000", 2024-09-11T08:43:29.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083800900000000002 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453980+0000", 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5509000000000001e-05 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:29.435 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.936855141999999, 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001121478, 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453572+0000", 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1600000000000001e-07 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453577+0000", 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4199999999999998e-06 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453588+0000", 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1134999999999999e-05 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453594+0000", 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453654+0000", 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9202e-05 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453668+0000", 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4375999999999999e-05 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454647+0000", 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097895199999999995 2024-09-11T08:43:29.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454693+0000", 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5558e-05 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.936302251000001, 2024-09-11T08:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001004079, 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454125+0000", 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2399999999999997e-07 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454212+0000", 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6881999999999999e-05 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454234+0000", 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2109000000000001e-05 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454251+0000", 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7552999999999999e-05 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454294+0000", 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2910000000000001e-05 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454306+0000", 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1853e-05 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455087+0000", 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078128799999999999 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455128+0000", 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.066e-05 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.934364874, 2024-09-11T08:43:29.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010907619999999999, 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456062+0000", 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7899999999999998e-07 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456134+0000", 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1617000000000002e-05 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456145+0000", 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1581e-05 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456156+0000", 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0671e-05 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456200+0000", 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4159999999999997e-05 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456217+0000", 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7061000000000001e-05 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457093+0000", 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087593800000000004 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457152+0000", 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9055000000000003e-05 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.926499124999999, 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-11T08:43:29.454 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.926076532, 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001143493, 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2799999999999996e-07 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464395+0000", 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4858000000000003e-05 2024-09-11T08:43:29.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464409+0000", 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3553000000000001e-05 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464416+0000", 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2899999999999997e-06 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464487+0000", 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0994000000000005e-05 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464520+0000", 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2790000000000003e-05 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465459+0000", 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093904100000000005 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465493+0000", 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4338999999999998e-05 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.459 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.913428393, 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001017771, 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7700000000000004e-07 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477043+0000", 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4397999999999997e-05 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477053+0000", 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0565e-05 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477063+0000", 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6169999999999996e-06 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477138+0000", 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5370000000000005e-05 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477151+0000", 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2962999999999999e-05 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477978+0000", 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082701499999999996 2024-09-11T08:43:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.478016+0000", 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7166000000000003e-05 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:99 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'65, mlcod=16'63)", 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.765717821999999, 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010323559999999999, 2024-09-11T08:43:29.464 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624715+0000", 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9739999999999996e-06 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624744+0000", 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8240000000000001e-05 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624783+0000", 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9555000000000003e-05 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624838+0000", 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4728000000000002e-05 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624927+0000", 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9493000000000005e-05 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624955+0000", 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7966999999999998e-05 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625702+0000", 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074682700000000004 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625741+0000", 2024-09-11T08:43:29.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8572000000000003e-05 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.762301033, 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010216209999999999, 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628126+0000", 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1699999999999998e-07 2024-09-11T08:43:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628133+0000", 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0639999999999996e-06 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628143+0000", 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628149+0000", 2024-09-11T08:43:29.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628197+0000", 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8272000000000001e-05 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628217+0000", 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0171999999999999e-05 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629111+0000", 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089439299999999995 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629147+0000", 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5317000000000001e-05 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-11T08:43:29.473 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.650031345999999, 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001056343, 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7899999999999999e-07 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740399+0000", 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1389999999999997e-06 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740411+0000", 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1313e-05 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740415+0000", 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1269999999999996e-06 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740462+0000", 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6659999999999997e-05 2024-09-11T08:43:29.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740475+0000", 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3301e-05 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741430+0000", 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095528300000000002 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741451+0000", 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1041000000000001e-05 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.649596192000001, 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099133399999999997, 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.478 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740831+0000", 2024-09-11T08:43:29.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740929+0000", 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8220000000000005e-05 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740943+0000", 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4121000000000001e-05 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740947+0000", 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3250000000000001e-06 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740990+0000", 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2842000000000001e-05 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741010+0000", 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0168000000000001e-05 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741802+0000", 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079172499999999996 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741821+0000", 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9434999999999999e-05 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.648499995000002, 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098299900000000003, 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741927+0000", 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3299999999999998e-07 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741932+0000", 2024-09-11T08:43:29.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741938+0000", 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6099999999999997e-06 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741941+0000", 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.27e-06 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741971+0000", 2024-09-11T08:43:29.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9708000000000001e-05 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741982+0000", 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1148000000000001e-05 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742880+0000", 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089861100000000001 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742909+0000", 2024-09-11T08:43:29.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8824999999999999e-05 2024-09-11T08:43:29.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.487 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.487 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.487 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:29.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-11T08:43:29.490 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:29.489+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:29.589 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:29.590+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:29.637 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:29.637 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:29.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:29.690 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:29.691+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:29.753 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:29.753 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:29.753 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:29.753 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:29.753 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:29.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:29.805 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:29.806+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:29.868 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 11.480090033, 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095305400000000005, 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7299999999999996e-07 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389734+0000", 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-05 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389772+0000", 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7768999999999997e-05 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389783+0000", 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839000000000001e-05 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389852+0000", 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9268000000000002e-05 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389880+0000", 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7597999999999999e-05 2024-09-11T08:43:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390619+0000", 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073945300000000005 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390671+0000", 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.475680767, 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095448600000000005, 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9399999999999999e-07 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394137+0000", 2024-09-11T08:43:29.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0331e-05 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394155+0000", 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7558999999999999e-05 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394170+0000", 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4591e-05 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:29.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394237+0000", 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7026999999999994e-05 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394257+0000", 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0145000000000001e-05 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395041+0000", 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078400400000000004 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395081+0000", 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0635e-05 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:29.878 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:29.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:29.920 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:29.920+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:29.992 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:29.993+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:30.027 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:30.027 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:30.027 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:30.027 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:30.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:30.093 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:30.093+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:30.144 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:30.144 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:30.144 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:30.144 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:30.144 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:30.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:30.194 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:30.195+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:30.295 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:30.294+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:30.301 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:30.301 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:30.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:30.301 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:30.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.918806485999999, 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025475459999999999, 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:30.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383750+0000", 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383766+0000", 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5752000000000001e-05 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383780+0000", 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4122e-05 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383881+0000", 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010105 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383914+0000", 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2999e-05 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386246+0000", 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023319579999999999 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:30.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386291+0000", 2024-09-11T08:43:30.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4435999999999999e-05 2024-09-11T08:43:30.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:30.306 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:30.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:30.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:30.306 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:30.306 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:30.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:30.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:30.395 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:30.396+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:30.463 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:30.463 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:30.463 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:30.463 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:30.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:30.479 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:30.496 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:30.496+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:30.497 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:30.497 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:30.515 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:30.515 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:30.533 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:30.533 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:30.550 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:30.550 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:30.567 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:30.568 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:30.582 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:30.582 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:30.585 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:30.591 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:30.597 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:30.597+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:30.603 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:30.698 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:30.697+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:30.709 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:30.709 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:30.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:30.798 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:30.799+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:30.833 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:30.833 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:30.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:30.926 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:30.927+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:30.986 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:30.986 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:30.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:31.001 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:31.001+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:31.102 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:31.102+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:31.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:31.203 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:31.203+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:31.248 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:31.248 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:31.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:31.304 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:31.304+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:31.358 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:31.358 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:31.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:31.405 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:31.404+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:31.424 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:43:31.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T08:43:31.470 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:31.470 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:31.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:31.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T08:43:31.535 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:31.535+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:31.583 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:31.584 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:31.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:31.635 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:31.636+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:31.696 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:31.696 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:31.703 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:31.744 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:31.744+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:31.807 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:31.808+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:31.845 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:31.845 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:31.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:31.908 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:31.908+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:31.996 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:31.996 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:32.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:32.009 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:32.009+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:32.107 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:32.107 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:32.109 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:32.110+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:32.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:32.210 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:32.211+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:32.219 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:32.219 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:32.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:32.311 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:32.312+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:32.342 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:32.342 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:32.351 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:32.413 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:32.413+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:32.428 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-11T08:43:32.431 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T08:43:32.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T08:43:32.463 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:32.463 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:32.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:32.513 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:32.513+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:32.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:32.615 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:32.615+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:32.622 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:32.622 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:32.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:32.716 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:32.716+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:32.744 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:32.764 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:32.764 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:32.781 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:32.781 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:32.785 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:32.785 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:32.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:32.799 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:32.799 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:32.815 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:32.816 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:32.816 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:32.815+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:32.833 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:32.833 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:32.850 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:32.850 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:32.867 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:32.907 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:32.908 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:32.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:32.920 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:32.917+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:33.017 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:33.017+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:33.065 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.676857415000001, 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095305400000000005, 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7299999999999996e-07 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389734+0000", 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-05 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389772+0000", 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7768999999999997e-05 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389783+0000", 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839000000000001e-05 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389852+0000", 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9268000000000002e-05 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389880+0000", 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7597999999999999e-05 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390619+0000", 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073945300000000005 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390671+0000", 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.672448149000001, 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095448600000000005, 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9399999999999999e-07 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394137+0000", 2024-09-11T08:43:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0331e-05 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394155+0000", 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7558999999999999e-05 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394170+0000", 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4591e-05 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394237+0000", 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7026999999999994e-05 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394257+0000", 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0145000000000001e-05 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395041+0000", 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078400400000000004 2024-09-11T08:43:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395081+0000", 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0635e-05 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:33.075 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:33.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:33.118 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:33.118+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:33.184 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:33.184 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:33.193 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:33.236 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:33.236+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:33.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:33.319 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:33.319+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:33.419 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:33.419+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T08:43:33.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T08:43:33.459 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.076845124999998, 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025475459999999999, 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:43:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383750+0000", 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383766+0000", 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5752000000000001e-05 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383780+0000", 2024-09-11T08:43:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4122e-05 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383881+0000", 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010105 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383914+0000", 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2999e-05 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386246+0000", 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023319579999999999 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386291+0000", 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4435999999999999e-05 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:33.464 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:33.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:33.552 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:33.552+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:33.621 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:33.620+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:33.640 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:33.640 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:33.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:33.732 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:33.732+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:33.796 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:33.796 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:33.797 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:33.805 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:33.847 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:33.846+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:33.924 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:33.925+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:33.953 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:33.953 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-11T08:43:33.953 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:33.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:34.026 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:34.026+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:34.076 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:34.076 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:34.076 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:34.077 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:34.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:34.127 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:34.127+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:34.228 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:34.228+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:34.230 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:34.230 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:34.238 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:34.329 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:34.330+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:34.341 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:34.341 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:34.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:34.341 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:34.341 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:34.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:34.433 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:34.434+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:34.441 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T08:43:34.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T08:43:34.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T08:43:34.493 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:34.493 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:34.493 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:34.493 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:34.501 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:34.542 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:34.543+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:34.631 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:34.631+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:34.645 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:34.645 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:34.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:34.732 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:34.733+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:34.758 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.377983531000002, 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:34.759 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:43:34.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:34.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:43:34.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:43:34.765 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.372798104000001, 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1896000000000003e-05, 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:34.766 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:34.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386389+0000", 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386394+0000", 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1839999999999998e-06 2024-09-11T08:43:34.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386405+0000", 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386412+0000", 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9530000000000002e-06 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386453+0000", 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.074e-05 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386460+0000", 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4479999999999997e-06 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386470+0000", 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-09-11T08:43:34.770 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.367708122, 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000246512, 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:34.771 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 2 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:34.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391480+0000", 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1129999999999999e-06 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391492+0000", 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1749999999999999e-05 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391522+0000", 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9886000000000002e-05 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391541+0000", 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9453000000000002e-05 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391624+0000", 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3307999999999996e-05 2024-09-11T08:43:34.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391724+0000", 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100002 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.367215054999999, 2024-09-11T08:43:34.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018776700000000001, 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:34.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391972+0000", 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391985+0000", 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3173000000000001e-05 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391995+0000", 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392003+0000", 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2449999999999998e-06 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392046+0000", 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2630999999999999e-05 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392159+0000", 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113069 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.371553791, 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:34.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:43:34.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:43:34.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.786 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.368415937000002, 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018276100000000001, 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390771+0000", 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4099999999999997e-07 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390778+0000", 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2230000000000001e-06 2024-09-11T08:43:34.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390789+0000", 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.081e-05 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390795+0000", 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9070000000000004e-06 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390869+0000", 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4010000000000005e-05 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390876+0000", 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390953+0000", 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6971000000000006e-05 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.790 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 15.367304605999999, 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:34.791 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:34.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:43:34.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:43:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 15.364016942999999, 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104585, 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395170+0000", 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1009999999999999e-06 2024-09-11T08:43:34.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395177+0000", 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395188+0000", 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0849999999999999e-05 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:34.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395194+0000", 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3999999999999997e-06 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395254+0000", 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0359000000000003e-05 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395261+0000", 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3679999999999998e-06 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:34.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395274+0000", 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.29e-05 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:34.802 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:34.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:34.833 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:34.833+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:34.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:34.934 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:34.933+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:34.946 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:34.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:35.003 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:35.021 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:35.035 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:35.036+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:35.037 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:35.053 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:35.058 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:35.058 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:35.058 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:35.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:35.070 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:35.087 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:35.104 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:35.104 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:35.121 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:35.121 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:35.150 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:35.151+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:35.210 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:35.210 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:35.219 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:35.237 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:35.237+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:35.338 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:35.339+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:35.364 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:35.364 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:35.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:35.439 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T08:43:35.442 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-11T08:43:35.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T08:43:35.457 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:35.457+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:35.516 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:35.516 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:35.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:35.541 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:35.541+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:35.641 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:35.640+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:35.668 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:35.668 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:35.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:35.742 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:35.741+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:35.784 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:35.784 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:35.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:35.843 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:35.844+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:35.944 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:35.944+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:35.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:36.045 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:36.046+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:36.097 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:36.097 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:36.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:36.149 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:36.149+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:36.214 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:36.214 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:36.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:36.263 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:36.263+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:36.348 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:36.349+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:36.370 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:36.370 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:36.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T08:43:36.449 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:36.449+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:36.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-11T08:43:36.489 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:36.489 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:36.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:36.550 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:36.551+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:36.651 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:36.652+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:36.668 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:36.668 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:36.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:36.752 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:36.752+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:36.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:36.852 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:36.852+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:36.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:36.953 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:36.953+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:37.053 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:37.054+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:37.058 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:37.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:37.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:37.154 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:37.154+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:37.175 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:37.185 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:37.259 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:37.259 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:37.258+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:37.293 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:37.293 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:37.312 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:37.312 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:37.328 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:37.328 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:37.348 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:37.349 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:37.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:37.349 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-11T08:43:37.349 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:37.356 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:37.357+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:37.358 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:37.363 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:37.363 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:37.380 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:37.380 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:37.397 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:37.449 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.6 to 1 (10000) 2024-09-11T08:43:37.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T08:43:37.458 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:37.458+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:37.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-11T08:43:37.475 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:37.476 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:37.476 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:37.476 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:37.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:37.558 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:37.558+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:37.630 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:37.630 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:37.630 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:37.630 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:37.630 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:37.639 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:37.683 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:37.684+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:37.742 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:37.742 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:37.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:37.743 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:37.743 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:37.751 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:37.760 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:37.760+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:37.861 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:37.862+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:37.896 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:37.896 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:37.896 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:37.896 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:37.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:37.962 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:37.962+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:38.048 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:38.048 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:38.048 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:38.048 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:38.048 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:38.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:38.101 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:38.101+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:38.160 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:38.160 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:38.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:38.160 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.779346046000001, 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:38.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:43:38.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:43:38.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:43:38.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.774160619, 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1896000000000003e-05, 2024-09-11T08:43:38.167 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386389+0000", 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386394+0000", 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1839999999999998e-06 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:38.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386405+0000", 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386412+0000", 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9530000000000002e-06 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386453+0000", 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.074e-05 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386460+0000", 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4479999999999997e-06 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386470+0000", 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:38.171 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.769070636999999, 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000246512, 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 2 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:38.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391480+0000", 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1129999999999999e-06 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391492+0000", 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1749999999999999e-05 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391522+0000", 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9886000000000002e-05 2024-09-11T08:43:38.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391541+0000", 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9453000000000002e-05 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391624+0000", 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3307999999999996e-05 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391724+0000", 2024-09-11T08:43:38.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100002 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.768577570000001, 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018776700000000001, 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:38.177 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:38.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391972+0000", 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391985+0000", 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3173000000000001e-05 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391995+0000", 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392003+0000", 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2449999999999998e-06 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392046+0000", 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2630999999999999e-05 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392159+0000", 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113069 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 19.772916305999999, 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:43:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:38.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:43:38.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 19.769778452000001, 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018276100000000001, 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:38.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390771+0000", 2024-09-11T08:43:38.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4099999999999997e-07 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390778+0000", 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2230000000000001e-06 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390789+0000", 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.081e-05 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390795+0000", 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9070000000000004e-06 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390869+0000", 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4010000000000005e-05 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390876+0000", 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390953+0000", 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6971000000000006e-05 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:38.192 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.768667121, 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:38.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:43:38.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:43:38.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.199 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.200 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-11T08:43:38.200 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:38.200 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.765379457999998, 2024-09-11T08:43:38.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104585, 2024-09-11T08:43:38.200 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:38.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395170+0000", 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1009999999999999e-06 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395177+0000", 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395188+0000", 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0849999999999999e-05 2024-09-11T08:43:38.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:38.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395194+0000", 2024-09-11T08:43:38.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3999999999999997e-06 2024-09-11T08:43:38.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:38.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395254+0000", 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0359000000000003e-05 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395261+0000", 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3679999999999998e-06 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395274+0000", 2024-09-11T08:43:38.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.29e-05 2024-09-11T08:43:38.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.205 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:38.205 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:38.205 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:38.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:38.207 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:38.206+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:38.307 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:38.308+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:38.350 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:38.350 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:38.351 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:38.351 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:38.359 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:38.408 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:38.408+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.7 to 1 (10000) 2024-09-11T08:43:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-11T08:43:38.476 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T08:43:38.476 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:43:38.504 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:38.504 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:38.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:38.519 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:38.519+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:38.610 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:38.611+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:38.615 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:38.615 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:38.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:38.615 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:38.615 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:38.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:38.711 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:38.712+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:38.768 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:38.768 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:38.768 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:38.769 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:38.777 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:38.813 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:38.813 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:43:38.813 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:38.812+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:38.825 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-11T08:43:38.459273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":208968,"kb_used_data":3100,"kb_used_omap":10,"kb_used_meta":205685,"kb_avail":660393912,"statfs":{"total":676457349120,"available":676243365888,"internally_reserved":0,"allocated":3174400,"data_stored":1907633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":210621547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017666"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.527965+0000","last_change":"2024-09-11T08:43:19.626333+0000","last_active":"2024-09-11T08:43:33.527965+0000","last_peered":"2024-09-11T08:43:33.527965+0000","last_clean":"2024-09-11T08:43:33.527965+0000","last_became_active":"2024-09-11T08:43:17.384965+0000","last_became_peered":"2024-09-11T08:43:17.384965+0000","last_unstale":"2024-09-11T08:43:33.527965+0000","last_undegraded":"2024-09-11T08:43:33.527965+0000","last_fullsized":"2024-09-11T08:43:33.527965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:09.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037094500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.527940+0000","last_change":"2024-09-11T08:43:26.983024+0000","last_active":"2024-09-11T08:43:33.527940+0000","last_peered":"2024-09-11T08:43:33.527940+0000","last_clean":"2024-09-11T08:43:33.527940+0000","last_became_active":"2024-09-11T08:43:26.982655+0000","last_became_peered":"2024-09-11T08:43:26.982655+0000","last_unstale":"2024-09-11T08:43:33.527940+0000","last_undegraded":"2024-09-11T08:43:33.527940+0000","last_fullsized":"2024-09-11T08:43:33.527940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:15:53.733553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.528011+0000","last_change":"2024-09-11T08:43:19.626282+0000","last_active":"2024-09-11T08:43:33.528011+0000","last_peered":"2024-09-11T08:43:33.528011+0000","last_clean":"2024-09-11T08:43:33.528011+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:33.528011+0000","last_undegraded":"2024-09-11T08:43:33.528011+0000","last_fullsized":"2024-09-11T08:43:33.528011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:10.938383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049065999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T08:43:34.441603+0000","last_change":"2024-09-11T08:43:26.979188+0000","last_active":"2024-09-11T08:43:34.441603+0000","last_peered":"2024-09-11T08:43:34.441603+0000","last_clean":"2024-09-11T08:43:34.441603+0000","last_became_active":"2024-09-11T08:43:26.978867+0000","last_became_peered":"2024-09-11T08:43:26.978867+0000","last_unstale":"2024-09-11T08:43:34.441603+0000","last_undegraded":"2024-09-11T08:43:34.441603+0000","last_fullsized":"2024-09-11T08:43:34.441603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:56:32.769126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.434666+0000","last_change":"2024-09-11T08:43:26.995270+0000","last_active":"2024-09-11T08:43:33.434666+0000","last_peered":"2024-09-11T08:43:33.434666+0000","last_clean":"2024-09-11T08:43:33.434666+0000","last_became_active":"2024-09-11T08:43:26.982695+0000","last_became_peered":"2024-09-11T08:43:26.982695+0000","last_unstale":"2024-09-11T08:43:33.434666+0000","last_undegraded":"2024-09-11T08:43:33.434666+0000","last_fullsized":"2024-09-11T08:43:33.434666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:51.215562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010925100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.527466+0000","last_change":"2024-09-11T08:43:19.469399+0000","last_active":"2024-09-11T08:43:33.527466+0000","last_peered":"2024-09-11T08:43:33.527466+0000","last_clean":"2024-09-11T08:43:33.527466+0000","last_became_active":"2024-09-11T08:43:17.383480+0000","last_became_peered":"2024-09-11T08:43:17.383480+0000","last_unstale":"2024-09-11T08:43:33.527466+0000","last_undegraded":"2024-09-11T08:43:33.527466+0000","last_fullsized":"2024-09-11T08:43:33.527466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:43.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040896800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.527964+0000","last_change":"2024-09-11T08:43:26.978762+0000","last_active":"2024-09-11T08:43:33.527964+0000","last_peered":"2024-09-11T08:43:33.527964+0000","last_clean":"2024-09-11T08:43:33.527964+0000","last_became_active":"2024-09-11T08:43:26.978432+0000","last_became_peered":"2024-09-11T08:43:26.978432+0000","last_unstale":"2024-09-11T08:43:33.527964+0000","last_undegraded":"2024-09-11T08:43:33.527964+0000","last_fullsized":"2024-09-11T08:43:33.527964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:12.237153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048316400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.434666+0000","last_change":"2024-09-11T08:43:19.394034+0000","last_active":"2024-09-11T08:43:33.434666+0000","last_peered":"2024-09-11T08:43:33.434666+0000","last_clean":"2024-09-11T08:43:33.434666+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:43:33.434666+0000","last_undegraded":"2024-09-11T08:43:33.434666+0000","last_fullsized":"2024-09-11T08:43:33.434666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:54.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719799999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.528097+0000","last_change":"2024-09-11T08:43:10.406953+0000","last_active":"2024-09-11T08:43:33.528097+0000","last_peered":"2024-09-11T08:43:33.528097+0000","last_clean":"2024-09-11T08:43:33.528097+0000","last_became_active":"2024-09-11T08:43:10.405743+0000","last_became_peered":"2024-09-11T08:43:10.405743+0000","last_unstale":"2024-09-11T08:43:33.528097+0000","last_undegraded":"2024-09-11T08:43:33.528097+0000","last_fullsized":"2024-09-11T08:43:33.528097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:41.277131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":688787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":98438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":98438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":270336,"data_stored":98419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509449,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":694205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":307200,"data_stored":114673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35656,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336184,"statfs":{"total":96636764160,"available":96600252416,"internally_reserved":0,"allocated":307200,"data_stored":114673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:38.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:43:38.913 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:38.913+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:38.920 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:38.920 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:38.920 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:38.920 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:38.921 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:38.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:39.014 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:39.014+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 29.672991564, 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:43:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:43:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.598889201999999, 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:43:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:43:39.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.598817713999999, 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:43:39.047 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:39.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:43:39.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:43:39.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.598794102999999, 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.054 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:39.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:43:39.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:43:39.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.55702672, 2024-09-11T08:43:39.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:39.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:43:39.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.557002854, 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:39.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:43:39.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:43:39.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:43:39.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:43:39.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.556896796, 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.074 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:39.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:43:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.40519432, 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:43:39.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:43:39.085 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.405165224000001, 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.086 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:39.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:43:39.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:43:39.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.405138885, 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:43:39.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:43:39.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:43:39.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.405069586, 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:43:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:43:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.405044633999999, 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:43:39.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.110 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.995347498000001, 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.111 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:43:39.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:43:39.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:43:39.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.995259884999999, 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:43:39.117 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:43:39.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.123 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.995220376999999, 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:43:39.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:43:39.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.995194172000001, 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:43:39.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:43:39.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.135 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.995142916999999, 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.136 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:39.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:43:39.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:43:39.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.995130998, 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:43:39.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:43:39.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.995117121, 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.148 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:43:39.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:43:39.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.154 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.995075579000002, 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:43:39.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:43:39.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:39.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:43:39.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:43:39.161 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.161 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.161 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.161 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:39.161 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:39.161 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:39.162 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:39.163 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:43:39.163 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:39.161+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:39.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:39.167 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-11T08:43:38.459273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":208968,"kb_used_data":3100,"kb_used_omap":10,"kb_used_meta":205685,"kb_avail":660393912,"statfs":{"total":676457349120,"available":676243365888,"internally_reserved":0,"allocated":3174400,"data_stored":1907633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":210621547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017666"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.527965+0000","last_change":"2024-09-11T08:43:19.626333+0000","last_active":"2024-09-11T08:43:33.527965+0000","last_peered":"2024-09-11T08:43:33.527965+0000","last_clean":"2024-09-11T08:43:33.527965+0000","last_became_active":"2024-09-11T08:43:17.384965+0000","last_became_peered":"2024-09-11T08:43:17.384965+0000","last_unstale":"2024-09-11T08:43:33.527965+0000","last_undegraded":"2024-09-11T08:43:33.527965+0000","last_fullsized":"2024-09-11T08:43:33.527965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:09.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037094500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.527940+0000","last_change":"2024-09-11T08:43:26.983024+0000","last_active":"2024-09-11T08:43:33.527940+0000","last_peered":"2024-09-11T08:43:33.527940+0000","last_clean":"2024-09-11T08:43:33.527940+0000","last_became_active":"2024-09-11T08:43:26.982655+0000","last_became_peered":"2024-09-11T08:43:26.982655+0000","last_unstale":"2024-09-11T08:43:33.527940+0000","last_undegraded":"2024-09-11T08:43:33.527940+0000","last_fullsized":"2024-09-11T08:43:33.527940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:15:53.733553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.528011+0000","last_change":"2024-09-11T08:43:19.626282+0000","last_active":"2024-09-11T08:43:33.528011+0000","last_peered":"2024-09-11T08:43:33.528011+0000","last_clean":"2024-09-11T08:43:33.528011+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:33.528011+0000","last_undegraded":"2024-09-11T08:43:33.528011+0000","last_fullsized":"2024-09-11T08:43:33.528011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:10.938383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049065999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T08:43:34.441603+0000","last_change":"2024-09-11T08:43:26.979188+0000","last_active":"2024-09-11T08:43:34.441603+0000","last_peered":"2024-09-11T08:43:34.441603+0000","last_clean":"2024-09-11T08:43:34.441603+0000","last_became_active":"2024-09-11T08:43:26.978867+0000","last_became_peered":"2024-09-11T08:43:26.978867+0000","last_unstale":"2024-09-11T08:43:34.441603+0000","last_undegraded":"2024-09-11T08:43:34.441603+0000","last_fullsized":"2024-09-11T08:43:34.441603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:56:32.769126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.434666+0000","last_change":"2024-09-11T08:43:26.995270+0000","last_active":"2024-09-11T08:43:33.434666+0000","last_peered":"2024-09-11T08:43:33.434666+0000","last_clean":"2024-09-11T08:43:33.434666+0000","last_became_active":"2024-09-11T08:43:26.982695+0000","last_became_peered":"2024-09-11T08:43:26.982695+0000","last_unstale":"2024-09-11T08:43:33.434666+0000","last_undegraded":"2024-09-11T08:43:33.434666+0000","last_fullsized":"2024-09-11T08:43:33.434666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:51.215562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010925100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.527466+0000","last_change":"2024-09-11T08:43:19.469399+0000","last_active":"2024-09-11T08:43:33.527466+0000","last_peered":"2024-09-11T08:43:33.527466+0000","last_clean":"2024-09-11T08:43:33.527466+0000","last_became_active":"2024-09-11T08:43:17.383480+0000","last_became_peered":"2024-09-11T08:43:17.383480+0000","last_unstale":"2024-09-11T08:43:33.527466+0000","last_undegraded":"2024-09-11T08:43:33.527466+0000","last_fullsized":"2024-09-11T08:43:33.527466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:43.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040896800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.527964+0000","last_change":"2024-09-11T08:43:26.978762+0000","last_active":"2024-09-11T08:43:33.527964+0000","last_peered":"2024-09-11T08:43:33.527964+0000","last_clean":"2024-09-11T08:43:33.527964+0000","last_became_active":"2024-09-11T08:43:26.978432+0000","last_became_peered":"2024-09-11T08:43:26.978432+0000","last_unstale":"2024-09-11T08:43:33.527964+0000","last_undegraded":"2024-09-11T08:43:33.527964+0000","last_fullsized":"2024-09-11T08:43:33.527964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:12.237153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048316400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.434666+0000","last_change":"2024-09-11T08:43:19.394034+0000","last_active":"2024-09-11T08:43:33.434666+0000","last_peered":"2024-09-11T08:43:33.434666+0000","last_clean":"2024-09-11T08:43:33.434666+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:43:33.434666+0000","last_undegraded":"2024-09-11T08:43:33.434666+0000","last_fullsized":"2024-09-11T08:43:33.434666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:54.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719799999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T08:43:33.528097+0000","last_change":"2024-09-11T08:43:10.406953+0000","last_active":"2024-09-11T08:43:33.528097+0000","last_peered":"2024-09-11T08:43:33.528097+0000","last_clean":"2024-09-11T08:43:33.528097+0000","last_became_active":"2024-09-11T08:43:10.405743+0000","last_became_peered":"2024-09-11T08:43:10.405743+0000","last_unstale":"2024-09-11T08:43:33.528097+0000","last_undegraded":"2024-09-11T08:43:33.528097+0000","last_fullsized":"2024-09-11T08:43:33.528097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:41.277131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":688787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":98438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":98438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":270336,"data_stored":98419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509449,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":694205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":307200,"data_stored":114673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35656,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336184,"statfs":{"total":96636764160,"available":96600252416,"internally_reserved":0,"allocated":307200,"data_stored":114673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:39.167 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T08:43:39.167 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:43:39.168 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-11T08:43:39.168 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:43:39.168 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:43:39.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-11T08:43:39.263 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:39.263+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:39.272 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:39.272 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:39.272 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:39.272 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:39.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:39.314 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:39.314 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T08:43:39.314 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:39.364 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:39.364+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:39.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:39.421 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:39.421 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:39.421 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:39.422 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:39.422 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:39.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:39.465 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:39.466+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:39.536 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:39.536 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:39.553 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:39.554 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:39.566 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:39.565+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:39.570 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:39.580 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:39.581 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:39.587 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:39.587 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:39.590 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:39.604 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:39.605 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:39.621 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:39.621 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:39.639 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:39.639 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:39.656 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:39.666 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:39.667+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:39.713 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:39.721 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:39.767 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:39.767+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:39.828 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:39.829 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:39.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:39.879 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:39.878+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:39.969 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:39.970+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:39.985 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:39.985 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:39.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:40.071 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:40.070+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:40.101 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:40.101 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:40.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:40.172 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:40.173+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:40.257 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:40.257 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:40.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:40.273 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:40.274+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:40.373 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:40.373 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:40.374 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:40.374+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:40.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:40.474 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:40.475+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:40.478 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T08:43:40.490 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:40.490 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:40.500 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:40.575 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:40.576+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:40.654 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:40.654 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:40.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:40.676 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:40.676+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:40.771 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:40.771 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:40.777 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:40.777+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:40.780 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:40.878 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:40.878+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:40.896 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:40.896 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:40.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:40.989 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:40.990+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:41.050 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:41.050 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:41.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:41.080 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:41.080+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:41.181 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:41.181+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:41.202 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:41.202 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:41.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:41.282 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:41.282+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:41.313 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:41.314 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:41.322 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:41.406 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:41.407+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:41.467 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:41.468 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:41.469 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-11T08:43:41.478 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:41.520 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:41.520+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:41.583 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:41.584+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:41.623 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:41.623 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:41.632 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:41.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:41.712 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:41.712+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:41.775 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:41.775 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:41.784 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:41.785 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:41.784+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:41.795 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:41.795 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:41.815 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:41.815 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:41.832 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:41.849 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:41.849 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:41.866 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:41.883 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:41.885 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:41.886+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:41.887 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:41.887 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:41.897 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:41.900 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:41.918 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:41.918 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:41.986 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:41.985+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:42.039 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:42.039 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:42.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:42.039 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:42.040 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:42.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:42.091 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:42.092+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:42.150 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:42.151 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:42.159 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:42.200 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:42.201+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:42.288 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:42.289+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:42.302 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:42.302 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:42.302 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:42.303 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:42.303 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:42.311 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:42.389 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:42.388+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.095989652999997, 2024-09-11T08:43:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:43:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:43:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:43:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 32.021887290999999, 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:43:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 32.021815803000003, 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:43:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:43:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 32.021792191999999, 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:43:42.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.980024809, 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.483 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:43:42.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.980000943, 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:43:42.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:43:42.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:43:42.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.979894885, 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.495 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:43:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:43:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.828192409, 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:43:42.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:43:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:43:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.828163313000001, 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:43:42.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:43:42.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.828136974, 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:43:42.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.519 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.828067675, 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:43:42.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:43:42.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:43:42.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.828042722999999, 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:43:42.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:43:42.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.418345587000001, 2024-09-11T08:43:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.418257973999999, 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:43:42.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:43:42.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.544 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.418218465999999, 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:43:42.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:43:42.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.418192261000002, 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.551 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:42.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:43:42.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.418141005999999, 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:43:42.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:43:42.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.418129087000001, 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:43:42.563 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:43:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:43:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:43:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.41811521, 2024-09-11T08:43:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:43:42.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.418073668000002, 2024-09-11T08:43:42.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:42.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:42.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:43:42.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:43:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:42.581 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:42.583 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:42.583+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:42.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:42.684 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:42.684+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:42.692 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:42.692 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:42.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:42.785 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:42.786+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:42.809 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:42.809 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:42.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:42.886 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:42.886+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:42.928 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.504585413000001, 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011043540000000001, 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423249+0000", 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0200000000000001e-06 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423401+0000", 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015192100000000001 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423432+0000", 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1282999999999998e-05 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423439+0000", 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.241e-06 2024-09-11T08:43:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423512+0000", 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3620000000000003e-05 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423529+0000", 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6813999999999999e-05 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424295+0000", 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076553700000000005 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424352+0000", 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6918e-05 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.493156630000001, 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011035800000000001, 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434677+0000", 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434683+0000", 2024-09-11T08:43:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.212e-06 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434701+0000", 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8067000000000002e-05 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434721+0000", 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9562999999999999e-05 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434778+0000", 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7581000000000003e-05 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434800+0000", 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2204e-05 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435738+0000", 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093753900000000004 2024-09-11T08:43:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435780+0000", 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1683000000000001e-05 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.492693271, 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:43:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:43:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:43:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.492173481999998, 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010649730000000001, 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435660+0000", 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.51e-07 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435665+0000", 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435674+0000", 2024-09-11T08:43:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:43:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435679+0000", 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6539999999999998e-06 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435815+0000", 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013680800000000001 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435838+0000", 2024-09-11T08:43:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2821000000000001e-05 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436692+0000", 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085374500000000005 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436724+0000", 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2246000000000003e-05 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:43:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.491629107999998, 2024-09-11T08:43:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:43:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:43:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'15)", 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.491135237999998, 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108423, 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436698+0000", 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4499999999999996e-07 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436737+0000", 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9226e-05 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436751+0000", 2024-09-11T08:43:42.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3172e-05 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436758+0000", 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689999999999999e-06 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436816+0000", 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7506999999999997e-05 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436836+0000", 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0324000000000001e-05 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437746+0000", 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000910537 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437782+0000", 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5150000000000001e-05 2024-09-11T08:43:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:38 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.486643190999999, 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010117800000000001, 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441190+0000", 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9999999999999996e-07 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441257+0000", 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6607999999999997e-05 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441269+0000", 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1953999999999999e-05 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441275+0000", 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9020000000000004e-06 2024-09-11T08:43:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441315+0000", 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0506999999999997e-05 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441328+0000", 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3165e-05 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442139+0000", 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081040300000000001 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442201+0000", 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2440999999999994e-05 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:39 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'25, mlcod=16'22)", 2024-09-11T08:43:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.486208976999997, 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010054160000000001, 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0900000000000002e-07 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441721+0000", 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6898000000000006e-05 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441734+0000", 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.332e-05 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441744+0000", 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.465e-06 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441812+0000", 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7694999999999995e-05 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441834+0000", 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2614000000000001e-05 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442593+0000", 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075926599999999998 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442629+0000", 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5648999999999997e-05 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:53 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'31, mlcod=16'29)", 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.474833431999997, 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098036799999999991, 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453000+0000", 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0400000000000005e-07 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453012+0000", 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1911e-05 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453024+0000", 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2257e-05 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453030+0000", 2024-09-11T08:43:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0009999999999998e-06 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453076+0000", 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5516000000000003e-05 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453096+0000", 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0361e-05 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453934+0000", 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083800900000000002 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453980+0000", 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5509000000000001e-05 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.474261558999999, 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001121478, 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453572+0000", 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1600000000000001e-07 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453577+0000", 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4199999999999998e-06 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453588+0000", 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1134999999999999e-05 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453594+0000", 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453654+0000", 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9202e-05 2024-09-11T08:43:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453668+0000", 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4375999999999999e-05 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454647+0000", 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097895199999999995 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454693+0000", 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5558e-05 2024-09-11T08:43:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.473708668, 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001004079, 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454125+0000", 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2399999999999997e-07 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454212+0000", 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6881999999999999e-05 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454234+0000", 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2109000000000001e-05 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454251+0000", 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7552999999999999e-05 2024-09-11T08:43:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454294+0000", 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2910000000000001e-05 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454306+0000", 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1853e-05 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455087+0000", 2024-09-11T08:43:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078128799999999999 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455128+0000", 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.066e-05 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.471771291000003, 2024-09-11T08:43:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010907619999999999, 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456062+0000", 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7899999999999998e-07 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456134+0000", 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1617000000000002e-05 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456145+0000", 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1581e-05 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456156+0000", 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0671e-05 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456200+0000", 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4159999999999997e-05 2024-09-11T08:43:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456217+0000", 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7061000000000001e-05 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457093+0000", 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087593800000000004 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457152+0000", 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9055000000000003e-05 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.463905541999999, 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:43:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.463482949000003, 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001143493, 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2799999999999996e-07 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464395+0000", 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4858000000000003e-05 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464409+0000", 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3553000000000001e-05 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464416+0000", 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2899999999999997e-06 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464487+0000", 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0994000000000005e-05 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464520+0000", 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2790000000000003e-05 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465459+0000", 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093904100000000005 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465493+0000", 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4338999999999998e-05 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.450834810000003, 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001017771, 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7700000000000004e-07 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477043+0000", 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4397999999999997e-05 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477053+0000", 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0565e-05 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477063+0000", 2024-09-11T08:43:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6169999999999996e-06 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477138+0000", 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5370000000000005e-05 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477151+0000", 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2962999999999999e-05 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477978+0000", 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082701499999999996 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.478016+0000", 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7166000000000003e-05 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:99 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'65, mlcod=16'63)", 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.303124238999999, 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010323559999999999, 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624715+0000", 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9739999999999996e-06 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624744+0000", 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8240000000000001e-05 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624783+0000", 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9555000000000003e-05 2024-09-11T08:43:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624838+0000", 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4728000000000002e-05 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624927+0000", 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9493000000000005e-05 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624955+0000", 2024-09-11T08:43:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7966999999999998e-05 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625702+0000", 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074682700000000004 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625741+0000", 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8572000000000003e-05 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.29970745, 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010216209999999999, 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628126+0000", 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1699999999999998e-07 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628133+0000", 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0639999999999996e-06 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628143+0000", 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628149+0000", 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628197+0000", 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8272000000000001e-05 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628217+0000", 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0171999999999999e-05 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629111+0000", 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089439299999999995 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629147+0000", 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5317000000000001e-05 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-11T08:43:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.187437762999998, 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001056343, 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7899999999999999e-07 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740399+0000", 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1389999999999997e-06 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740411+0000", 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1313e-05 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740415+0000", 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1269999999999996e-06 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740462+0000", 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6659999999999997e-05 2024-09-11T08:43:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740475+0000", 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3301e-05 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741430+0000", 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095528300000000002 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741451+0000", 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1041000000000001e-05 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.187002609, 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099133399999999997, 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740831+0000", 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740929+0000", 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8220000000000005e-05 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740943+0000", 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4121000000000001e-05 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740947+0000", 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3250000000000001e-06 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740990+0000", 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2842000000000001e-05 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741010+0000", 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0168000000000001e-05 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741802+0000", 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079172499999999996 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741821+0000", 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9434999999999999e-05 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.185906412000001, 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098299900000000003, 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741927+0000", 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3299999999999998e-07 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741932+0000", 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741938+0000", 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6099999999999997e-06 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741941+0000", 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.27e-06 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741971+0000", 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9708000000000001e-05 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741982+0000", 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1148000000000001e-05 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742880+0000", 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089861100000000001 2024-09-11T08:43:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742909+0000", 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8824999999999999e-05 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.024 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:43.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:43.026 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:43.026+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:43.127 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:43.127+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:43.179 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:43.179 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:43.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:43.230 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:43.230+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:43.329 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:43.330+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:43.337 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:43.337 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:43.337 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:43.337 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:43.337 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:43.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:43.431 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:43.431+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:43.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T08:43:43.496 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:43.496 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:43.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:43.496 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:43.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 25.107668465, 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095305400000000005, 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7299999999999996e-07 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389734+0000", 2024-09-11T08:43:43.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-05 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389772+0000", 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7768999999999997e-05 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389783+0000", 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839000000000001e-05 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389852+0000", 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9268000000000002e-05 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389880+0000", 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7597999999999999e-05 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390619+0000", 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073945300000000005 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390671+0000", 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 24.103259199, 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095448600000000005, 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:43.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9399999999999999e-07 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394137+0000", 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0331e-05 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394155+0000", 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7558999999999999e-05 2024-09-11T08:43:43.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394170+0000", 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4591e-05 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394237+0000", 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7026999999999994e-05 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394257+0000", 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0145000000000001e-05 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395041+0000", 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078400400000000004 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395081+0000", 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0635e-05 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.506 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.506 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:43.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:43.532 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:43.532+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:43.632 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:43.633+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:43.665 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:43.665 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:43.666 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:43.666 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:43.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:43.758 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:43.759+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:43.823 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:43.823 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:43.824 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:43.824 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:43.824 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:43.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:43.833 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:43.834+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:43.930 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:43.976 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:43.977+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:43.983 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:43.983 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 26.600979908999999, 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025475459999999999, 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:43.984 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383750+0000", 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383766+0000", 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5752000000000001e-05 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383780+0000", 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4122e-05 2024-09-11T08:43:43.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383881+0000", 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010105 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383914+0000", 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2999e-05 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386246+0000", 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023319579999999999 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386291+0000", 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4435999999999999e-05 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:43.989 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:43.990 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:43.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:44.035 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:44.035+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:44.059 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:44.060 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:44.078 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:44.096 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:44.114 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:44.114 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:44.133 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:44.135 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:44.136+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:44.148 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:44.149 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:44.150 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:44.150 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:44.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:44.168 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:44.169 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:44.188 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:44.236 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:44.236+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:44.279 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:44.279 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:44.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:44.323 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:43:44.324 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:43:44.324 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-11T08:43:44.324 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-11T08:43:44.336 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:44.336+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:44.403 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:44.403 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:44.413 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:44.459 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:44.460+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-11T08:43:44.531 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:44.531 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:44.538 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:44.538+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:44.539 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:44.639 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:44.640+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:44.653 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-11T08:43:44.653+0000 7f1a550d0640 -1 mon.a@0(leader).osd e33 definitely_dead 0 2024-09-11T08:43:44.684 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:44.684 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:44.692 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:44.740 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:44.740+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:44.795 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:44.795 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:44.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:44.845 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:44.845+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:44.941 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:44.942+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:44.946 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:44.946 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:44.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:45.042 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:45.043+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:45.098 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:45.098 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:45.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:45.147 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:45.148+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:45.244 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:45.244+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:45.250 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:45.250 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:45.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:45.345 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:45.346+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:45.362 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:45.362 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:45.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:45.446 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:45.447+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:marked in osd.1. 2024-09-11T08:43:45.492 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-11T08:43:45.516 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:45.517 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:45.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:45.547 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:45.547+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:45.628 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:45.628 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:45.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:45.647 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:45.648+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:45.739 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:45.739 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:45.748 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:45.748 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:45.748+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:45.849 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:45.849+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:45.891 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:45.892 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:45.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:45.950 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:45.950+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:46.003 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:46.003 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:46.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:46.052 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:46.053+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:46.120 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:46.121 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:46.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:46.152 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:46.153+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:46.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:46.243 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:46.243 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:46.253 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:46.253 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:46.253+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:46.333 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:46.333 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:46.351 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:46.353 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:46.354+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:46.369 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:46.385 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:46.402 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:46.402 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:46.416 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:46.416 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:46.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.992272493999998, 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011043540000000001, 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423249+0000", 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0200000000000001e-06 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423401+0000", 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015192100000000001 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423432+0000", 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1282999999999998e-05 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423439+0000", 2024-09-11T08:43:46.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.241e-06 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423512+0000", 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3620000000000003e-05 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423529+0000", 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6813999999999999e-05 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424295+0000", 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076553700000000005 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424352+0000", 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6918e-05 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.980843710999999, 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011035800000000001, 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434677+0000", 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434683+0000", 2024-09-11T08:43:46.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.212e-06 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434701+0000", 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8067000000000002e-05 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434721+0000", 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9562999999999999e-05 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434778+0000", 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7581000000000003e-05 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434800+0000", 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2204e-05 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435738+0000", 2024-09-11T08:43:46.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093753900000000004 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435780+0000", 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1683000000000001e-05 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:46.426 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.980380351999997, 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.979860563000003, 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010649730000000001, 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435660+0000", 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.51e-07 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435665+0000", 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435674+0000", 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435679+0000", 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6539999999999998e-06 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435815+0000", 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013680800000000001 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435838+0000", 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2821000000000001e-05 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436692+0000", 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085374500000000005 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436724+0000", 2024-09-11T08:43:46.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2246000000000003e-05 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.979316189000002, 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.439 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:46.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:43:46.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'15)", 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.978822319000002, 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108423, 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436698+0000", 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4499999999999996e-07 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436737+0000", 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9226e-05 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436751+0000", 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3172e-05 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436758+0000", 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689999999999999e-06 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436816+0000", 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7506999999999997e-05 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436836+0000", 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0324000000000001e-05 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437746+0000", 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000910537 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437782+0000", 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5150000000000001e-05 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.448 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:38 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.974330272000003, 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010117800000000001, 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441190+0000", 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9999999999999996e-07 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441257+0000", 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6607999999999997e-05 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441269+0000", 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1953999999999999e-05 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441275+0000", 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9020000000000004e-06 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441315+0000", 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0506999999999997e-05 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441328+0000", 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3165e-05 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442139+0000", 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081040300000000001 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442201+0000", 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2440999999999994e-05 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:39 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'25, mlcod=16'22)", 2024-09-11T08:43:46.453 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.973896058000001, 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010054160000000001, 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0900000000000002e-07 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441721+0000", 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6898000000000006e-05 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441734+0000", 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.332e-05 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441744+0000", 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.465e-06 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441812+0000", 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7694999999999995e-05 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441834+0000", 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2614000000000001e-05 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442593+0000", 2024-09-11T08:43:46.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075926599999999998 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442629+0000", 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5648999999999997e-05 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:53 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'31, mlcod=16'29)", 2024-09-11T08:43:46.458 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.962520513000001, 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098036799999999991, 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453000+0000", 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0400000000000005e-07 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453012+0000", 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1911e-05 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453024+0000", 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2257e-05 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453030+0000", 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0009999999999998e-06 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453076+0000", 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5516000000000003e-05 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453096+0000", 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0361e-05 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453934+0000", 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083800900000000002 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453980+0000", 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5509000000000001e-05 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-11T08:43:46.463 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.961948640000003, 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001121478, 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453572+0000", 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1600000000000001e-07 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453577+0000", 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4199999999999998e-06 2024-09-11T08:43:46.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453588+0000", 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1134999999999999e-05 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453594+0000", 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453654+0000", 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9202e-05 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453668+0000", 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4375999999999999e-05 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454647+0000", 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097895199999999995 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454693+0000", 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5558e-05 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-11T08:43:46.468 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.961395748999998, 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001004079, 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454125+0000", 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2399999999999997e-07 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454212+0000", 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6881999999999999e-05 2024-09-11T08:43:46.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454234+0000", 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2109000000000001e-05 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454251+0000", 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7552999999999999e-05 2024-09-11T08:43:46.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454294+0000", 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2910000000000001e-05 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454306+0000", 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1853e-05 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455087+0000", 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078128799999999999 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455128+0000", 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.066e-05 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-11T08:43:46.473 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.959458372, 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010907619999999999, 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456062+0000", 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7899999999999998e-07 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456134+0000", 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1617000000000002e-05 2024-09-11T08:43:46.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456145+0000", 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1581e-05 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456156+0000", 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0671e-05 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456200+0000", 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4159999999999997e-05 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456217+0000", 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7061000000000001e-05 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457093+0000", 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087593800000000004 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457152+0000", 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9055000000000003e-05 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.951592623000003, 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:43:46.478 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:43:46.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.95117003, 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001143493, 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2799999999999996e-07 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464395+0000", 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4858000000000003e-05 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464409+0000", 2024-09-11T08:43:46.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3553000000000001e-05 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464416+0000", 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2899999999999997e-06 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464487+0000", 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0994000000000005e-05 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464520+0000", 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2790000000000003e-05 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465459+0000", 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093904100000000005 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465493+0000", 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4338999999999998e-05 2024-09-11T08:43:46.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.938521891000001, 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001017771, 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7700000000000004e-07 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477043+0000", 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4397999999999997e-05 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477053+0000", 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0565e-05 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477063+0000", 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6169999999999996e-06 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477138+0000", 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5370000000000005e-05 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477151+0000", 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2962999999999999e-05 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477978+0000", 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082701499999999996 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.478016+0000", 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7166000000000003e-05 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:99 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'65, mlcod=16'63)", 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.790811320000003, 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010323559999999999, 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624715+0000", 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9739999999999996e-06 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624744+0000", 2024-09-11T08:43:46.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8240000000000001e-05 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624783+0000", 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9555000000000003e-05 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624838+0000", 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4728000000000002e-05 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624927+0000", 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9493000000000005e-05 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624955+0000", 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7966999999999998e-05 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625702+0000", 2024-09-11T08:43:46.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074682700000000004 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625741+0000", 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8572000000000003e-05 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:46.497 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.787394530999997, 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010216209999999999, 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628126+0000", 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1699999999999998e-07 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628133+0000", 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0639999999999996e-06 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628143+0000", 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628149+0000", 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628197+0000", 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8272000000000001e-05 2024-09-11T08:43:46.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628217+0000", 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0171999999999999e-05 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629111+0000", 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089439299999999995 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629147+0000", 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5317000000000001e-05 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 34.675124844000003, 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001056343, 2024-09-11T08:43:46.502 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7899999999999999e-07 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740399+0000", 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1389999999999997e-06 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740411+0000", 2024-09-11T08:43:46.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1313e-05 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740415+0000", 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1269999999999996e-06 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740462+0000", 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6659999999999997e-05 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740475+0000", 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3301e-05 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741430+0000", 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095528300000000002 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741451+0000", 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1041000000000001e-05 2024-09-11T08:43:46.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 34.674689690000001, 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099133399999999997, 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.507 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740831+0000", 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740929+0000", 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8220000000000005e-05 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740943+0000", 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4121000000000001e-05 2024-09-11T08:43:46.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740947+0000", 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3250000000000001e-06 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740990+0000", 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2842000000000001e-05 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741010+0000", 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0168000000000001e-05 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741802+0000", 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079172499999999996 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741821+0000", 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9434999999999999e-05 2024-09-11T08:43:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 34.673593492999998, 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098299900000000003, 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741927+0000", 2024-09-11T08:43:46.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3299999999999998e-07 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741932+0000", 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741938+0000", 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6099999999999997e-06 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741941+0000", 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.27e-06 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741971+0000", 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9708000000000001e-05 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741982+0000", 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1148000000000001e-05 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742880+0000", 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089861100000000001 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742909+0000", 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8824999999999999e-05 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.516 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:46.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T08:43:46.518 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:46.518 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:46.519+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:46.519 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:46.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:46.520 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:46.520 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:46.520 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:46.521 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:46.619 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:46.620+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:46.671 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:46.671 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:46.671 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:46.671 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:46.682 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:46.719 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:46.720+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:46.792 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:46.792 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:46.792 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:46.792 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:46.792 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:46.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:46.820 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:46.821+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:46.921 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:46.921+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:46.946 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:46.946 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:46.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:46.946 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.557296269999998, 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095305400000000005, 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7299999999999996e-07 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389734+0000", 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-05 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389772+0000", 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7768999999999997e-05 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389783+0000", 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839000000000001e-05 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389852+0000", 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9268000000000002e-05 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389880+0000", 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7597999999999999e-05 2024-09-11T08:43:46.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390619+0000", 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073945300000000005 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390671+0000", 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 27.552887003999999, 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095448600000000005, 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9399999999999999e-07 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394137+0000", 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0331e-05 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394155+0000", 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7558999999999999e-05 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394170+0000", 2024-09-11T08:43:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4591e-05 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394237+0000", 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7026999999999994e-05 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394257+0000", 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0145000000000001e-05 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395041+0000", 2024-09-11T08:43:46.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078400400000000004 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395081+0000", 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0635e-05 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(2.2 35/34 [PushOp(2:5cea7035:::rbd_info:head, version: 21'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@21'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:43:46.956 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.45478365100000001, 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044003299999999999, 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:46.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492231+0000", 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9400000000000005e-07 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492245+0000", 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4115e-05 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492265+0000", 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9585e-05 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492282+0000", 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7322000000000001e-05 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492387+0000", 2024-09-11T08:43:46.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010562599999999999 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492396+0000", 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4889999999999995e-06 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492670+0000", 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027430199999999998 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:46.961 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:46.961 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:46.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:47.022 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:47.022+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:47.123 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:47.124+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:47.131 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:47.132 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:47.132 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:47.132 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:47.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:47.224 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:47.225+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:47.325 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:47.325+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:47.357 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:47.357 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:47.357 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:47.357 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:47.357 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:47.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:47.425 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:47.426+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-11T08:43:47.527 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:47.527+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:47.567 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 30.184794783000001, 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025475459999999999, 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:43:47.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383750+0000", 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383766+0000", 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5752000000000001e-05 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:47.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383780+0000", 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4122e-05 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383881+0000", 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010105 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383914+0000", 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2999e-05 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386246+0000", 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023319579999999999 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386291+0000", 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4435999999999999e-05 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:47.572 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:47.579 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:47.627 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:47.628+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:47.710 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:47.710 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:47.710 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:47.710 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:47.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:47.763 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:47.764+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:47.829 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:47.830+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:47.846 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:47.846 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:47.846 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:47.846 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:47.846 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:47.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:47.930 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:47.931+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:48.007 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:48.007 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:48.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:48.007 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-11T08:43:48.007 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:48.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:48.031 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:48.031+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:48.131 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:48.132+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:48.132 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:48.132 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:48.133 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:48.133 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:48.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:48.232 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:48.233+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:48.287 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:48.287 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:48.287 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:48.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:48.333 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:48.334+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:48.434 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:48.435+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:48.438 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:48.438 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:48.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:48.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:48.534 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:48.535+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:48.590 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:48.590 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:48.591 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:48.599 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:48.640 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:48.641+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:48.648 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:48.666 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:48.666 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:48.683 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:48.701 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:48.719 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:48.736 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:48.737+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:48.736 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:48.737 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:48.742 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:48.742 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:48.752 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:48.754 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:48.754 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:48.772 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:48.772 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:48.837 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:48.837+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:48.854 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:48.855 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:48.863 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:48.938 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:48.938+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:49.007 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:49.007 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:49.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:49.059 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:49.060+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:49.118 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:49.118 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:49.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:49.140 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:49.140+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:49.241 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:49.241+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:49.269 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:49.269 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:49.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:49.342 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:49.343+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:49.422 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:49.422 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:49.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:49.443 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:49.444+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:49.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T08:43:49.534 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:49.535 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:49.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:49.552 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:49.553+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:49.646 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:49.646+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:49.646 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:49.647 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:49.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:49.747 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:49.747+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:49.772 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:49.772 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:49.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:49.847 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:49.847+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:49.897 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:49.898 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:49.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:49.948 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:49.949+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:50.015 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:50.015 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:50.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:50.067 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:50.068+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:50.133 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:50.133 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:50.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:50.150 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:50.151+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:50.251 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:50.252+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:50.300 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:50.300 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:50.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:50.352 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:50.353+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:50.419 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:50.419 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:50.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:50.471 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:50.472+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:50.493 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:43:50.493 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:43:50.493 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T08:43:50.493 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 1.000000 2024-09-11T08:43:50.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-11T08:43:50.499 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-11T08:43:50.546 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:50.547 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:50.553 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:50.553+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:50.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:50.654 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:50.655+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:50.713 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:50.713 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:50.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:50.766 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:50.767+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:50.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:50.845 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:50.845 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:50.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:50.855 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:50.856+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:50.921 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:50.954 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:50.956 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:50.957+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:50.985 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:50.990 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:50.990 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:51.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:51.003 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:51.003 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:51.023 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:51.042 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:51.057 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:51.057+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:51.061 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:51.080 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:51.114 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:51.114 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:51.114 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:51.115 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:51.115 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:51.123 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:51.164 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:51.165+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:51.258 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:51.259+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:51.273 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:51.273 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:51.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:51.273 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-11T08:43:51.273 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:51.283 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:51.359 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:51.359+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:51.399 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:51.399 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:51.400 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:51.400 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:51.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:51.494 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:51.494+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:51.549 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-11T08:43:51.554 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:51.555 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:51.561 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:51.561+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:51.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:51.662 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:51.663+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:51.707 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:51.707 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:51.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:51.707 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:51.707 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:51.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:51.799 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:51.800+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:51.859 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:51.860 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:51.860 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:51.860 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:51.864 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:51.864+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:51.868 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:51.964 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:51.965+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:52.012 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:52.012 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:52.012 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:52.012 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:52.012 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:52.020 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:52.066 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:52.066+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.744180874000001, 2024-09-11T08:43:52.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:52.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:43:52.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.132 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.738995447000001, 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1896000000000003e-05, 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386389+0000", 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386394+0000", 2024-09-11T08:43:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1839999999999998e-06 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386405+0000", 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386412+0000", 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9530000000000002e-06 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386453+0000", 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.074e-05 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386460+0000", 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4479999999999997e-06 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386470+0000", 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-09-11T08:43:52.136 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.733905464999999, 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000246512, 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:52.137 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 2 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391480+0000", 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1129999999999999e-06 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391492+0000", 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1749999999999999e-05 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391522+0000", 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9886000000000002e-05 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391541+0000", 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9453000000000002e-05 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391624+0000", 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3307999999999996e-05 2024-09-11T08:43:52.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391724+0000", 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100002 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.733412397999999, 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018776700000000001, 2024-09-11T08:43:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391972+0000", 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391985+0000", 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3173000000000001e-05 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391995+0000", 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392003+0000", 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2449999999999998e-06 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392046+0000", 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2630999999999999e-05 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392159+0000", 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113069 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.737751134, 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:43:52.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:43:52.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:43:52.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:43:52.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:43:52.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.734613279999998, 2024-09-11T08:43:52.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018276100000000001, 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390771+0000", 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4099999999999997e-07 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390778+0000", 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2230000000000001e-06 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390789+0000", 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.081e-05 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390795+0000", 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9070000000000004e-06 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390869+0000", 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4010000000000005e-05 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390876+0000", 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390953+0000", 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6971000000000006e-05 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 32.733501949000001, 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:43:52.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:43:52.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 32.730214285999999, 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104585, 2024-09-11T08:43:52.167 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395170+0000", 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1009999999999999e-06 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395177+0000", 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395188+0000", 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0849999999999999e-05 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395194+0000", 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3999999999999997e-06 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395254+0000", 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0359000000000003e-05 2024-09-11T08:43:52.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395261+0000", 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3679999999999998e-06 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395274+0000", 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.29e-05 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.171 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(2.2 35/34 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.6320464279999998, 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000180452, 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3599999999999999e-07 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:52.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493342+0000", 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493350+0000", 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8990000000000001e-06 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493359+0000", 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8759999999999994e-06 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493397+0000", 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8303999999999998e-05 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493517+0000", 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012013199999999999 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:52.175 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:52.176 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:52.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:52.178 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:52.178+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:52.278 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:52.279+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:52.321 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:52.322 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:52.322 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:52.322 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:52.330 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:52.379 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:52.379+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:52.433 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:52.433 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:52.433 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:52.433 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:52.434 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:52.442 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:52.479 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:52.480+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:52.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T08:43:52.581 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:52.581+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:52.587 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:52.587 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:52.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:52.587 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:52.587 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:52.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:52.681 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:52.682+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:52.739 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:52.739 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:52.747 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:52.791 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:52.792+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:52.849 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:52.850 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:52.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:52.900 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:52.900+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:52.983 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:52.984+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.641849121, 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:43:53.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:43:53.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.567746759000002, 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:43:53.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:43:53.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.567675270999999, 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.016 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:43:53.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:43:53.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:43:53.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.567651660000003, 2024-09-11T08:43:53.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.525884277000003, 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:43:53.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.525860410999996, 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:43:53.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.525754352999996, 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.041 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:43:53.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:43:53.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.374051876999999, 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.047 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:53.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:43:53.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:43:53.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.053 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.374022781000001, 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:53.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:43:53.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:43:53.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:43:53.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:43:53.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:53.060 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.373996441999999, 2024-09-11T08:43:53.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:43:53.061 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:53.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:43:53.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.067 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.373927143000003, 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:43:53.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:43:53.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.373902190999999, 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.074 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:43:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:43:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 41.964205055000001, 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:43:53.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:43:53.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 41.964117442000003, 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:43:53.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:43:53.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 41.964077934000002, 2024-09-11T08:43:53.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 41.964051728999998, 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:53.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:43:53.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 41.964000474000002, 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:43:53.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:43:53.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 41.963988555, 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:43:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:43:53.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:43:53.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:43:53.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.118 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.118 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.119 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:53.119 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 41.963974678, 2024-09-11T08:43:53.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:43:53.119 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:43:53.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.125 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 41.963933136000001, 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.126 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:53.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:43:53.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:53.131 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:53.132 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:53.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:53.133 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:53.133+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:53.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:53.234 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:53.234+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:53.248 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:53.248 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:53.248 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:53.248 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:53.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:53.263 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:53.264 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:53.283 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:53.284 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:53.301 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:53.318 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:53.318 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:53.334 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:53.335+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:53.335 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:53.353 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:53.371 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:53.379 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:53.379 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:53.388 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:53.388 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:53.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:53.435 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:53.436+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-11T08:43:53.536 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:53.537+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:53.557 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:53.557 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:53.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:53.651 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:53.652+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:53.717 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:53.717 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:53.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:53.768 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:53.768+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:53.839 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:53.840+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:53.892 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:53.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:53.945 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:53.946+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:54.041 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:54.042+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:54.065 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:54.066 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:54.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:54.142 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:54.143+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:54.186 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:54.187 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:54.196 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:54.243 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:54.244+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:54.311 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:54.311 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:54.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:54.344 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:54.345+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:54.445 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:54.446+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:54.471 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:54.471 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:54.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:54.546 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:54.546+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:54.591 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:54.591 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:54.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:54.684 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:54.685+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:54.747 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:54.747+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:54.751 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:54.751 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:54.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:54.848 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:54.849+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:54.912 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:54.913 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:54.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:54.962 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:54.963+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:55.037 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:55.037 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:55.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:55.089 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:55.090+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:55.149 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:55.149 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:55.151 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:55.152+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:55.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:55.252 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:55.252+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:55.303 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:55.303 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:55.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:55.354 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:55.354+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:55.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:55.454 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:55.455+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:55.456 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:55.456 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:55.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:55.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T08:43:55.536 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:55.555 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:55.556+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:55.560 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:55.567 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:55.567 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:55.576 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:55.578 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:55.595 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:55.596 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:55.612 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:55.630 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:55.630 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:55.647 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:55.656 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:55.656+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:55.678 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 38.300342649999997, 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:43:55.681 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:43:55.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:43:55.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 38.295157222999997, 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1896000000000003e-05, 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386389+0000", 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386394+0000", 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1839999999999998e-06 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386405+0000", 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-09-11T08:43:55.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386412+0000", 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9530000000000002e-06 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386453+0000", 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.074e-05 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386460+0000", 2024-09-11T08:43:55.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4479999999999997e-06 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386470+0000", 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 38.290067241000003, 2024-09-11T08:43:55.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000246512, 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 2 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391480+0000", 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1129999999999999e-06 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391492+0000", 2024-09-11T08:43:55.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1749999999999999e-05 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391522+0000", 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9886000000000002e-05 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391541+0000", 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9453000000000002e-05 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391624+0000", 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3307999999999996e-05 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391724+0000", 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100002 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 38.289574174000002, 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018776700000000001, 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391972+0000", 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-09-11T08:43:55.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391985+0000", 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3173000000000001e-05 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391995+0000", 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392003+0000", 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2449999999999998e-06 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392046+0000", 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2630999999999999e-05 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392159+0000", 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113069 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.700 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.293912910000003, 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:43:55.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:43:55.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.290775056000001, 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018276100000000001, 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390771+0000", 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4099999999999997e-07 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390778+0000", 2024-09-11T08:43:55.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2230000000000001e-06 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390789+0000", 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.081e-05 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390795+0000", 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9070000000000004e-06 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390869+0000", 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4010000000000005e-05 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390876+0000", 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390953+0000", 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6971000000000006e-05 2024-09-11T08:43:55.711 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.289663724999997, 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:55.712 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:43:55.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:43:55.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.718 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.286376062000002, 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104585, 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395170+0000", 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1009999999999999e-06 2024-09-11T08:43:55.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395177+0000", 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395188+0000", 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0849999999999999e-05 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395194+0000", 2024-09-11T08:43:55.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3999999999999997e-06 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395254+0000", 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0359000000000003e-05 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395261+0000", 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3679999999999998e-06 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395274+0000", 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.29e-05 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(2.2 35/34 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.1882082040000004, 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000180452, 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:55.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:55.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3599999999999999e-07 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493342+0000", 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493350+0000", 2024-09-11T08:43:55.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8990000000000001e-06 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493359+0000", 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8759999999999994e-06 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493397+0000", 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8303999999999998e-05 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:55.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493517+0000", 2024-09-11T08:43:55.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012013199999999999 2024-09-11T08:43:55.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.727 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:55.727 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:55.727 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:55.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:55.771 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:55.772+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:55.859 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:55.859+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:55.873 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:55.874 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:55.874 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:55.874 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:55.882 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:55.960 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:55.959+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:55.985 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:55.985 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:55.986 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:55.986 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:55.986 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:55.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:43:56.081 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:56.081+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:56.139 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:56.139 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:56.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:56.139 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:43:56.139 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:56.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:43:56.161 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:56.162+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:56.262 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:56.262+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:56.290 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:56.290 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:56.291 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:43:56.291 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:56.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:43:56.363 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:56.362+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:56.402 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:56.402 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:43:56.402 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:43:56.402 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:43:56.402 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:56.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:43:56.464 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:56.465+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:56.520 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-11T08:43:56.550 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:43:56.551 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:43:56.551 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T08:43:56.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T08:43:56.559 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 47.197132207000003, 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:43:56.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.561 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:56.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:43:56.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:43:56.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:43:56.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:43:56.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:43:56.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:43:56.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:43:56.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:43:56.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.123029844999998, 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:43:56.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.571 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:56.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:43:56.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:43:56.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:43:56.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:43:56.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:43:56.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:43:56.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:43:56.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.577 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.122958357000002, 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:43:56.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:43:56.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:43:56.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:43:56.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:43:56.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:43:56.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:43:56.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:43:56.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:43:56.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:43:56.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.122934745999999, 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:43:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:43:56.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:43:56.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.593 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.081167362999999, 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:43:56.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.595 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:56.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:43:56.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:43:56.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:43:56.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:43:56.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:43:56.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:43:56.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:43:56.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.601 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.081143496999999, 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.602 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:56.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:56.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:43:56.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:43:56.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:43:56.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:43:56.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:43:56.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:43:56.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:43:56.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:43:56.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:43:56.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:43:56.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:43:56.610 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.610 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.610 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.081037438999999, 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:43:56.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:43:56.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:43:56.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:43:56.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:43:56.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:43:56.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:43:56.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:43:56.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:43:56.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:43:56.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:43:56.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:43:56.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:43:56.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:43:56.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:43:56.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:43:56.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:43:56.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:43:56.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.929334963000002, 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:43:56.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:43:56.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:43:56.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:43:56.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:43:56.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:43:56.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:43:56.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:43:56.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:43:56.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:43:56.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:43:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.929305866999997, 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.630 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.630 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.630 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:43:56.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.630 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:56.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:56.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:43:56.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:43:56.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:43:56.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:43:56.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:43:56.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:43:56.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:43:56.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:43:56.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:43:56.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:56.638 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.929279528000002, 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:43:56.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:43:56.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:43:56.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:43:56.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:43:56.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:43:56.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:43:56.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:43:56.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.646 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.647 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.647 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:56.647 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.929210228999999, 2024-09-11T08:43:56.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:43:56.647 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:43:56.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:43:56.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:43:56.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:43:56.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:43:56.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.657 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.929185277000002, 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:43:56.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:43:56.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.663 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.519488140999997, 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:43:56.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.669 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.519400527999998, 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:43:56.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.675 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.519361019999998, 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:43:56.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:43:56.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.681 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.519334815000001, 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:43:56.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:56.687 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.519283559999998, 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:56.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:43:56.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:56.693 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.519271641000003, 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:43:56.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.519257764000002, 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:43:56.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:43:56.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.519216221999997, 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:43:56.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:43:56.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:43:56.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.711 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:43:56.712 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:43:56.712 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:43:56.713 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:43:56.714 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:56.713+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:56.814 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:56.815+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:56.842 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:56.842 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:56.842 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:56.842 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:56.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:43:56.915 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:56.915+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:57.016 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:57.016+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:57.040 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:57.040 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:57.040 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:57.040 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:57.041 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:57.053 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:43:57.117 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:57.118+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:57.186 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:57.186 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:43:57.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.761739554999998, 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011043540000000001, 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423249+0000", 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0200000000000001e-06 2024-09-11T08:43:57.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423401+0000", 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015192100000000001 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423432+0000", 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1282999999999998e-05 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423439+0000", 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.241e-06 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423512+0000", 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3620000000000003e-05 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423529+0000", 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6813999999999999e-05 2024-09-11T08:43:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424295+0000", 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076553700000000005 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424352+0000", 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6918e-05 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.750310771999999, 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011035800000000001, 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434677+0000", 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434683+0000", 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.212e-06 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434701+0000", 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8067000000000002e-05 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434721+0000", 2024-09-11T08:43:57.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9562999999999999e-05 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434778+0000", 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7581000000000003e-05 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434800+0000", 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2204e-05 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435738+0000", 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093753900000000004 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435780+0000", 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1683000000000001e-05 2024-09-11T08:43:57.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.749847412999998, 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:43:57.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.201 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.749327624000003, 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010649730000000001, 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435660+0000", 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.51e-07 2024-09-11T08:43:57.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435665+0000", 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435674+0000", 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435679+0000", 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6539999999999998e-06 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435815+0000", 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013680800000000001 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435838+0000", 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2821000000000001e-05 2024-09-11T08:43:57.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436692+0000", 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085374500000000005 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436724+0000", 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2246000000000003e-05 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.748783250000002, 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:43:57.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'15)", 2024-09-11T08:43:57.211 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.748289380000003, 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108423, 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:57.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436698+0000", 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4499999999999996e-07 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436737+0000", 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9226e-05 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436751+0000", 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3172e-05 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436758+0000", 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689999999999999e-06 2024-09-11T08:43:57.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436816+0000", 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7506999999999997e-05 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436836+0000", 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0324000000000001e-05 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437746+0000", 2024-09-11T08:43:57.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000910537 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437782+0000", 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5150000000000001e-05 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:38 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.743797333000003, 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010117800000000001, 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:57.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:43:57.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441190+0000", 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9999999999999996e-07 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441257+0000", 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6607999999999997e-05 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441269+0000", 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1953999999999999e-05 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441275+0000", 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9020000000000004e-06 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441315+0000", 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0506999999999997e-05 2024-09-11T08:43:57.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441328+0000", 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3165e-05 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442139+0000", 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081040300000000001 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442201+0000", 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2440999999999994e-05 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:39 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'25, mlcod=16'22)", 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.743363119000001, 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010054160000000001, 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0900000000000002e-07 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.224 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441721+0000", 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6898000000000006e-05 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441734+0000", 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.332e-05 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441744+0000", 2024-09-11T08:43:57.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.465e-06 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441812+0000", 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7694999999999995e-05 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441834+0000", 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2614000000000001e-05 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442593+0000", 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075926599999999998 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442629+0000", 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5648999999999997e-05 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:53 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'31, mlcod=16'29)", 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.731987574000001, 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098036799999999991, 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453000+0000", 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0400000000000005e-07 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453012+0000", 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1911e-05 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453024+0000", 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2257e-05 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453030+0000", 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0009999999999998e-06 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453076+0000", 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5516000000000003e-05 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453096+0000", 2024-09-11T08:43:57.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0361e-05 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453934+0000", 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083800900000000002 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453980+0000", 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5509000000000001e-05 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.232 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.731415701000003, 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001121478, 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:57.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453572+0000", 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1600000000000001e-07 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453577+0000", 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4199999999999998e-06 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453588+0000", 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1134999999999999e-05 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453594+0000", 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453654+0000", 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9202e-05 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453668+0000", 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4375999999999999e-05 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454647+0000", 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097895199999999995 2024-09-11T08:43:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454693+0000", 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5558e-05 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.730862809999998, 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001004079, 2024-09-11T08:43:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454125+0000", 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2399999999999997e-07 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454212+0000", 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6881999999999999e-05 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454234+0000", 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2109000000000001e-05 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454251+0000", 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7552999999999999e-05 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454294+0000", 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2910000000000001e-05 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454306+0000", 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1853e-05 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455087+0000", 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078128799999999999 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455128+0000", 2024-09-11T08:43:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.066e-05 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.728925433000001, 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010907619999999999, 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456062+0000", 2024-09-11T08:43:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7899999999999998e-07 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456134+0000", 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1617000000000002e-05 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456145+0000", 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1581e-05 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456156+0000", 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0671e-05 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456200+0000", 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4159999999999997e-05 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456217+0000", 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7061000000000001e-05 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457093+0000", 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087593800000000004 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457152+0000", 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9055000000000003e-05 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.721059683999997, 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:43:57.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:43:57.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.720637091, 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001143493, 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2799999999999996e-07 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464395+0000", 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4858000000000003e-05 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464409+0000", 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3553000000000001e-05 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464416+0000", 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2899999999999997e-06 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464487+0000", 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0994000000000005e-05 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464520+0000", 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2790000000000003e-05 2024-09-11T08:43:57.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465459+0000", 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093904100000000005 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465493+0000", 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4338999999999998e-05 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.257 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.707988952000001, 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001017771, 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7700000000000004e-07 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477043+0000", 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4397999999999997e-05 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477053+0000", 2024-09-11T08:43:57.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0565e-05 2024-09-11T08:43:57.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477063+0000", 2024-09-11T08:43:57.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6169999999999996e-06 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477138+0000", 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5370000000000005e-05 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477151+0000", 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2962999999999999e-05 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477978+0000", 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082701499999999996 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.478016+0000", 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7166000000000003e-05 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.266 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:99 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'65, mlcod=16'63)", 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.560278381000003, 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010323559999999999, 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624715+0000", 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9739999999999996e-06 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624744+0000", 2024-09-11T08:43:57.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8240000000000001e-05 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624783+0000", 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9555000000000003e-05 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624838+0000", 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4728000000000002e-05 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624927+0000", 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9493000000000005e-05 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624955+0000", 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7966999999999998e-05 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625702+0000", 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074682700000000004 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625741+0000", 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8572000000000003e-05 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.271 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.556861591999997, 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010216209999999999, 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628126+0000", 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1699999999999998e-07 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628133+0000", 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0639999999999996e-06 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628143+0000", 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628149+0000", 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628197+0000", 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8272000000000001e-05 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628217+0000", 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0171999999999999e-05 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629111+0000", 2024-09-11T08:43:57.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089439299999999995 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629147+0000", 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5317000000000001e-05 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-11T08:43:57.276 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 45.444591905000003, 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001056343, 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7899999999999999e-07 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740399+0000", 2024-09-11T08:43:57.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1389999999999997e-06 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740411+0000", 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1313e-05 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740415+0000", 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1269999999999996e-06 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740462+0000", 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6659999999999997e-05 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740475+0000", 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3301e-05 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741430+0000", 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095528300000000002 2024-09-11T08:43:57.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741451+0000", 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1041000000000001e-05 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-11T08:43:57.281 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 45.444156751000001, 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099133399999999997, 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740831+0000", 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-11T08:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740929+0000", 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8220000000000005e-05 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740943+0000", 2024-09-11T08:43:57.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4121000000000001e-05 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740947+0000", 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3250000000000001e-06 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740990+0000", 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2842000000000001e-05 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741010+0000", 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0168000000000001e-05 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741802+0000", 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079172499999999996 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741821+0000", 2024-09-11T08:43:57.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9434999999999999e-05 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 45.443060553999999, 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098299900000000003, 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:43:57.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741927+0000", 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3299999999999998e-07 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741932+0000", 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741938+0000", 2024-09-11T08:43:57.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6099999999999997e-06 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741941+0000", 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.27e-06 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741971+0000", 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9708000000000001e-05 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741982+0000", 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1148000000000001e-05 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742880+0000", 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089861100000000001 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742909+0000", 2024-09-11T08:43:57.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8824999999999999e-05 2024-09-11T08:43:57.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.292 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.292 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.292 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 40, 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "75755954-09e8-48d3-b9c1-83899efaff39", 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-11T08:42:57.181838+0000", 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-11T08:43:56.514765+0000", 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: "last_up_change": "2024-09-11T08:43:09.343672+0000", 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: "last_in_change": "2024-09-11T08:43:45.477548+0000", 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T08:43:57.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_num": 5804032, 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_set": [ 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "nodeep-scrub", 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "pglog_hardlimit", 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "purged_snapdirs", 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_deletes", 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "sortbitwise" 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_version": 5, 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_snapshot": "", 2024-09-11T08:43:57.294 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_max": 2, 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd": 8, 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "require_min_compat_client": "luminous", 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min_compat_client": "jewel", 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "require_osd_release": "squid", 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_crimson": false, 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "pools": [ 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 1, 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": ".mgr", 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-11T08:43:08.201248+0000", 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool", 2024-09-11T08:43:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 1, 2024-09-11T08:43:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 1, 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 1, 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 1, 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 1, 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-11T08:43:57.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "17", 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 0, 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 0, 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-11T08:43:57.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-11T08:43:57.299 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-11T08:43:57.300 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "options": { 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_max": 32, 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_min": 1 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": {} 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 8, 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 8, 2024-09-11T08:43:57.301 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.25, 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 2, 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 2, 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 2, 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": "rbd", 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-11T08:43:16.189979+0000", 2024-09-11T08:43:57.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 8193, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:43:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 8, 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 8, 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 8, 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 8, 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 8, 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-11T08:43:57.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "21", 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 2, 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 21, 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-11T08:43:57.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-11T08:43:57.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-11T08:43:57.307 INFO:teuthology.orchestra.run.smithi045.stdout: "options": {}, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd": {} 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 1.9900000095367432, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 1.9900000095367432, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.87999999523162842, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 1.75, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 1.75, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-11T08:43:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "ac6b86cf-4ee8-49eb-a680-a350b1c9879c", 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:43:57.309 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 22, 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6808", 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:43:57.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6809", 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6810", 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:43:57.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6811", 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6814", 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6815", 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.313 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6812", 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6813", 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.314 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6809/2540486849", 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6811/2540486849", 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6815/2540486849", 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6813/2540486849", 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "e27d776b-f77b-4111-a536-5f5d8030c696", 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 34, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:43:57.316 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6824", 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6825", 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.318 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:43:57.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.318 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6826", 2024-09-11T08:43:57.318 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:43:57.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6827", 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6830", 2024-09-11T08:43:57.319 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6831", 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6828", 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6829", 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6825/1040844445", 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6827/1040844445", 2024-09-11T08:43:57.321 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6831/1040844445", 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6829/1040844445", 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "6b8e44f6-395b-447d-a5d3-c19a0a4be61b", 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:43:57.322 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 18, 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6816", 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6817", 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.324 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6818", 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6819", 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6822", 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6823", 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:43:57.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6820", 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6821", 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.327 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6817/1231172466", 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6819/1231172466", 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6823/1231172466", 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6821/1231172466", 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "1d1e1983-4c89-4cb4-a6b4-af244f941a7a", 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 13, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 0, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:43:57.329 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6802", 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6803", 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6806", 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6807", 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6804", 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:43:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6805", 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6801/226138367", 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6803/226138367", 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6807/226138367", 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6805/226138367", 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:43:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "fd9239ee-3a74-4399-af2f-1e9a63a2f642", 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:43:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 13, 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 0, 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6800", 2024-09-11T08:43:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6801", 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6802", 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6803", 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:43:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6806", 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6807", 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:43:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6804", 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6805", 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6801/374760845", 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6803/374760845", 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6807/374760845", 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6805/374760845", 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-11T08:43:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "cfff2555-5ad4-4629-a725-afadf9b6a96e", 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 34, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6824", 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6825", 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6826", 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6827", 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:43:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6830", 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6831", 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:43:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6828", 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6829", 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.346 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6825/498386186", 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6827/498386186", 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6831/498386186", 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6829/498386186", 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.347 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "4588b13e-af90-48a7-a10d-f7a185a902e2", 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 22, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:43:57.348 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6808", 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6809", 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.349 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6810", 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6811", 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.350 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6814", 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6815", 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:43:57.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6812", 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6813", 2024-09-11T08:43:57.352 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6809/1261869816", 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6811/1261869816", 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6815/1261869816", 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6813/1261869816", 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:43:57.353 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "12da1d21-9c59-4462-9d5a-61801dab3e4d", 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 35, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:43:57.354 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6816", 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6817", 2024-09-11T08:43:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6818", 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.356 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6819", 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6822", 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:43:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6823", 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6820", 2024-09-11T08:43:57.358 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6821", 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6817/2975777507", 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6819/2975777507", 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2975777507", 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2975777507", 2024-09-11T08:43:57.359 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:43:57.360 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:43:57.360 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:43:57.360 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:43:57.360 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.360 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:43:57.360 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_xinfo": [ 2024-09-11T08:43:57.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.360 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-11T08:43:57.361 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:43:57.361 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:43:57.361 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:43:57.361 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:43:57.361 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:43:57.361 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.394370+0000", 2024-09-11T08:43:57.361 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:43:57.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.813228+0000", 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:43:57.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.425286+0000", 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-11T08:43:57.363 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:43:57.364 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:43:57.364 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:43:57.364 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:43:57.364 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:43:57.364 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.147591+0000", 2024-09-11T08:43:57.364 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:43:57.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:07.119248+0000", 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:43:57.365 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.653838+0000", 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:43:57.366 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.459308+0000", 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.492589+0000", 2024-09-11T08:43:57.367 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap": [], 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_items": [], 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_primaries": [], 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_temp": [], 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_temp": [], 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklist": {}, 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "range_blocklist": {}, 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profiles": { 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default": { 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-failure-domain": "osd", 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "k": "2", 2024-09-11T08:43:57.368 INFO:teuthology.orchestra.run.smithi045.stdout: "m": "1", 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin": "jerasure", 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "technique": "reed_sol_van" 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps_queue": [], 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "new_removed_snaps": [], 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "new_purged_snaps": [], 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_node_flags": {}, 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "device_class_flags": {}, 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": { 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_enabled": false, 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_bucket_count": 0, 2024-09-11T08:43:57.369 INFO:teuthology.orchestra.run.smithi045.stdout: "degraded_stretch_mode": 0, 2024-09-11T08:43:57.370 INFO:teuthology.orchestra.run.smithi045.stdout: "recovering_stretch_mode": 0, 2024-09-11T08:43:57.370 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_bucket": 0 2024-09-11T08:43:57.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:43:57.370 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:57.371 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:43:57.372 INFO:tasks.thrashosds.thrasher:j is {'epoch': 40, 'fsid': '75755954-09e8-48d3-b9c1-83899efaff39', 'created': '2024-09-11T08:42:57.181838+0000', 'modified': '2024-09-11T08:43:56.514765+0000', 'last_up_change': '2024-09-11T08:43:09.343672+0000', 'last_in_change': '2024-09-11T08:43:45.477548+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:43:08.201248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T08:43:16.189979+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ac6b86cf-4ee8-49eb-a680-a350b1c9879c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6808', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6809', 'nonce': 2540486849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6810', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6811', 'nonce': 2540486849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6814', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6815', 'nonce': 2540486849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6812', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6813', 'nonce': 2540486849}]}, 'public_addr': '172.21.15.45:6809/2540486849', 'cluster_addr': '172.21.15.45:6811/2540486849', 'heartbeat_back_addr': '172.21.15.45:6815/2540486849', 'heartbeat_front_addr': '172.21.15.45:6813/2540486849', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e27d776b-f77b-4111-a536-5f5d8030c696', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6824', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6825', 'nonce': 1040844445}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6826', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6827', 'nonce': 1040844445}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6830', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6831', 'nonce': 1040844445}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6828', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6829', 'nonce': 1040844445}]}, 'public_addr': '172.21.15.45:6825/1040844445', 'cluster_addr': '172.21.15.45:6827/1040844445', 'heartbeat_back_addr': '172.21.15.45:6831/1040844445', 'heartbeat_front_addr': '172.21.15.45:6829/1040844445', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '6b8e44f6-395b-447d-a5d3-c19a0a4be61b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6816', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6817', 'nonce': 1231172466}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6818', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6819', 'nonce': 1231172466}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6822', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6823', 'nonce': 1231172466}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6820', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6821', 'nonce': 1231172466}]}, 'public_addr': '172.21.15.45:6817/1231172466', 'cluster_addr': '172.21.15.45:6819/1231172466', 'heartbeat_back_addr': '172.21.15.45:6823/1231172466', 'heartbeat_front_addr': '172.21.15.45:6821/1231172466', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1d1e1983-4c89-4cb4-a6b4-af244f941a7a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6800', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6801', 'nonce': 226138367}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6802', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6803', 'nonce': 226138367}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6806', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6807', 'nonce': 226138367}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6804', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6805', 'nonce': 226138367}]}, 'public_addr': '172.21.15.45:6801/226138367', 'cluster_addr': '172.21.15.45:6803/226138367', 'heartbeat_back_addr': '172.21.15.45:6807/226138367', 'heartbeat_front_addr': '172.21.15.45:6805/226138367', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd9239ee-3a74-4399-af2f-1e9a63a2f642', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6800', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6801', 'nonce': 374760845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6802', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6803', 'nonce': 374760845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6806', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6807', 'nonce': 374760845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6804', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6805', 'nonce': 374760845}]}, 'public_addr': '172.21.15.86:6801/374760845', 'cluster_addr': '172.21.15.86:6803/374760845', 'heartbeat_back_addr': '172.21.15.86:6807/374760845', 'heartbeat_front_addr': '172.21.15.86:6805/374760845', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'cfff2555-5ad4-4629-a725-afadf9b6a96e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6824', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6825', 'nonce': 498386186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6826', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6827', 'nonce': 498386186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6830', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6831', 'nonce': 498386186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6828', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6829', 'nonce': 498386186}]}, 'public_addr': '172.21.15.86:6825/498386186', 'cluster_addr': '172.21.15.86:6827/498386186', 'heartbeat_back_addr': '172.21.15.86:6831/498386186', 'heartbeat_front_addr': '172.21.15.86:6829/498386186', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '4588b13e-af90-48a7-a10d-f7a185a902e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6808', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6809', 'nonce': 1261869816}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6810', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6811', 'nonce': 1261869816}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6814', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6815', 'nonce': 1261869816}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6812', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6813', 'nonce': 1261869816}]}, 'public_addr': '172.21.15.86:6809/1261869816', 'cluster_addr': '172.21.15.86:6811/1261869816', 'heartbeat_back_addr': '172.21.15.86:6815/1261869816', 'heartbeat_front_addr': '172.21.15.86:6813/1261869816', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '12da1d21-9c59-4462-9d5a-61801dab3e4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6816', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6817', 'nonce': 2975777507}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6818', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6819', 'nonce': 2975777507}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6822', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6823', 'nonce': 2975777507}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6820', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6821', 'nonce': 2975777507}]}, 'public_addr': '172.21.15.86:6817/2975777507', 'cluster_addr': '172.21.15.86:6819/2975777507', 'heartbeat_back_addr': '172.21.15.86:6823/2975777507', 'heartbeat_front_addr': '172.21.15.86:6821/2975777507', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.394370+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.813228+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.425286+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.147591+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:07.119248+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.653838+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.459308+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.492589+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T08:43:57.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:43:57.374 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:57.374+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:57.475 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:57.475+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:57.571 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:57.571 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:57.571 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:43:57.572 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:57.576 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:57.576+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:57.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:43:57.676 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:57.677+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:57.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:57.746 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:43:57.746 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:43:57.746 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:43:57.746 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:43:57.746 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:43:57.758 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:43:57.777 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:57.777+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:57.877 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:57.877+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:57.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:43:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:43:57.898 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:43:57.898 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:57.907 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-11T08:43:56.520139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":241524,"kb_used_data":4492,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733196,"statfs":{"total":773094113280,"available":772846792704,"internally_reserved":0,"allocated":4599808,"data_stored":2516656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.020574"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.728602+0000","last_change":"2024-09-11T08:43:19.626333+0000","last_active":"2024-09-11T08:43:53.728602+0000","last_peered":"2024-09-11T08:43:53.728602+0000","last_clean":"2024-09-11T08:43:53.728602+0000","last_became_active":"2024-09-11T08:43:17.384965+0000","last_became_peered":"2024-09-11T08:43:17.384965+0000","last_unstale":"2024-09-11T08:43:53.728602+0000","last_undegraded":"2024-09-11T08:43:53.728602+0000","last_fullsized":"2024-09-11T08:43:53.728602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:09.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037094500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.511412+0000","last_change":"2024-09-11T08:43:46.492417+0000","last_active":"2024-09-11T08:43:53.511412+0000","last_peered":"2024-09-11T08:43:53.511412+0000","last_clean":"2024-09-11T08:43:53.511412+0000","last_became_active":"2024-09-11T08:43:46.492204+0000","last_became_peered":"2024-09-11T08:43:46.492204+0000","last_unstale":"2024-09-11T08:43:53.511412+0000","last_undegraded":"2024-09-11T08:43:53.511412+0000","last_fullsized":"2024-09-11T08:43:53.511412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:43.614795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.728676+0000","last_change":"2024-09-11T08:43:19.626282+0000","last_active":"2024-09-11T08:43:53.728676+0000","last_peered":"2024-09-11T08:43:53.728676+0000","last_clean":"2024-09-11T08:43:53.728676+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:43:53.728676+0000","last_undegraded":"2024-09-11T08:43:53.728676+0000","last_fullsized":"2024-09-11T08:43:53.728676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:10.938383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049065999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.511337+0000","last_change":"2024-09-11T08:43:46.864029+0000","last_active":"2024-09-11T08:43:53.511337+0000","last_peered":"2024-09-11T08:43:53.511337+0000","last_clean":"2024-09-11T08:43:53.511337+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:43:53.511337+0000","last_undegraded":"2024-09-11T08:43:53.511337+0000","last_fullsized":"2024-09-11T08:43:53.511337+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:51.854546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.512029+0000","last_change":"2024-09-11T08:43:46.493691+0000","last_active":"2024-09-11T08:43:53.512029+0000","last_peered":"2024-09-11T08:43:53.512029+0000","last_clean":"2024-09-11T08:43:53.512029+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:43:53.512029+0000","last_undegraded":"2024-09-11T08:43:53.512029+0000","last_fullsized":"2024-09-11T08:43:53.512029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:14.454230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010925100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.728256+0000","last_change":"2024-09-11T08:43:19.469399+0000","last_active":"2024-09-11T08:43:53.728256+0000","last_peered":"2024-09-11T08:43:53.728256+0000","last_clean":"2024-09-11T08:43:53.728256+0000","last_became_active":"2024-09-11T08:43:17.383480+0000","last_became_peered":"2024-09-11T08:43:17.383480+0000","last_unstale":"2024-09-11T08:43:53.728256+0000","last_undegraded":"2024-09-11T08:43:53.728256+0000","last_fullsized":"2024-09-11T08:43:53.728256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:43.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040896800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.728657+0000","last_change":"2024-09-11T08:43:47.895970+0000","last_active":"2024-09-11T08:43:53.728657+0000","last_peered":"2024-09-11T08:43:53.728657+0000","last_clean":"2024-09-11T08:43:53.728657+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:43:53.728657+0000","last_undegraded":"2024-09-11T08:43:53.728657+0000","last_fullsized":"2024-09-11T08:43:53.728657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:23:07.574724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048316400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":55,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.512044+0000","last_change":"2024-09-11T08:43:19.394034+0000","last_active":"2024-09-11T08:43:53.512044+0000","last_peered":"2024-09-11T08:43:53.512044+0000","last_clean":"2024-09-11T08:43:53.512044+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:43:53.512044+0000","last_undegraded":"2024-09-11T08:43:53.512044+0000","last_fullsized":"2024-09-11T08:43:53.512044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:54.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719799999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T08:43:53.728727+0000","last_change":"2024-09-11T08:43:10.406953+0000","last_active":"2024-09-11T08:43:53.728727+0000","last_peered":"2024-09-11T08:43:53.728727+0000","last_clean":"2024-09-11T08:43:53.728727+0000","last_became_active":"2024-09-11T08:43:10.405743+0000","last_became_peered":"2024-09-11T08:43:10.405743+0000","last_unstale":"2024-09-11T08:43:53.728727+0000","last_undegraded":"2024-09-11T08:43:53.728727+0000","last_fullsized":"2024-09-11T08:43:53.728727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:19:41.277131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":759395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":169027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":169046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":169027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":434176,"data_stored":169046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1024000,"data_stored":759395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":158135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35760,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336080,"statfs":{"total":96636764160,"available":96600145920,"internally_reserved":0,"allocated":417792,"data_stored":163585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:57.907 INFO:tasks.thrashosds.thrasher:Setting 1.0 to [2, 0] 2024-09-11T08:43:57.908 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.0', '2', '0'] 2024-09-11T08:43:57.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 1.0 2 0 2024-09-11T08:43:57.922 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:43:57.922 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:57.954 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:43:57.954 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:57.973 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:57.974 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:57.978 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:57.977+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:57.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:43:57.989 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:43:57.989 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:58.018 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:43:58.018 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:58.037 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:43:58.037 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:58.070 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:43:58.071 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:58.079 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:58.078+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:58.103 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:43:58.104 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:58.143 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:58.143 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:58.156 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:43:58.198 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:58.198+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:58.280 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:58.280+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:43:58.321 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:58.321 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:58.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:43:58.415 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:43:58.415+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:43:58.482 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:58.482+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:58.493 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:58.494 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:58.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:43:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:set 1.0 pg_upmap mapping to [2,0] 2024-09-11T08:43:58.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T08:43:58.584 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:43:58.583+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:43:58.638 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:58.638 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:58.647 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:43:58.684 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:58.684+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:58.785 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:58.785+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:58.807 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:58.807 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:58.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:43:58.886 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:43:58.886+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:43:58.944 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:58.945 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:58.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:43:58.999 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:58.998+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:59.069 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:59.069 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:59.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:43:59.120 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:59.119+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:59.189 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:59.188+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:59.223 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:59.224 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:59.232 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:43:59.290 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:59.291+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:59.375 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:59.375 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:59.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:43:59.391 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:43:59.392+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:43:59.486 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:59.487 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:59.492 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:59.492+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:59.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-11T08:43:59.593 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:43:59.592+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:43:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:59.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:43:59.693 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:59.694+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:59.792 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:59.792 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:59.794 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:43:59.795+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:43:59.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:43:59.895 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:43:59.895+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:43:59.904 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:59.904 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:59.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:43:59.996 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:43:59.997+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:00.055 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:00.055 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:00.064 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:00.097 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:00.097+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:00.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:00.166 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:00.166 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:00.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:00.219 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:00.219+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:00.279 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:00.279 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:00.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:00.296 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:00.296 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:00.312 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:00.330 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:00.329+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:00.330 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:00.346 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:00.364 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:00.380 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:00.399 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:00.398+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:00.408 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:00.409 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:00.434 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:00.434 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:00.434 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:00.434 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:00.434 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:00.442 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:00.526 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:00.526+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 51.228938100000001, 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:44:00.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:44:00.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 50.154835738000003, 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:44:00.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:44:00.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 50.154764249999999, 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:44:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:00.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.609 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 50.154740639000003, 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:44:00.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.615 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 50.112973255999997, 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:44:00.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:44:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:44:00.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.621 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 50.112949389999997, 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:44:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:44:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 50.112843331999997, 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:44:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.961140856, 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:44:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:44:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.961111760000001, 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.640 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:00.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:44:00.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:44:00.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.961085421, 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:44:00.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:44:00.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.961016121999997, 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:44:00.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:44:00.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.96099117, 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:44:00.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.551294034000001, 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:44:00.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:44:00.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.551206421000003, 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:44:00.670 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:44:00.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:44:00.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.676 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.551166913000003, 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:44:00.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:44:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.682 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.551140707999998, 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:44:00.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:44:00.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.551089453000003, 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:44:00.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:44:00.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:44:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.694 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.551077534000001, 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:44:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:44:00.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.551063657, 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:44:00.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.551022115000002, 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:00.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:44:00.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:44:00.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:00.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:00.713 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:00.715 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:00.715+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:00.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:00.816 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:00.816+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:00.880 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:00.881 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:00.881 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:00.881 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:00.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:00.933 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:00.934+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:01.017 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:01.018+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:01.066 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:01.066 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:01.066 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:01.066 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:01.066 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:01.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:01.118 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:01.119+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:01.219 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:01.219+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.805761347999997, 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011043540000000001, 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423249+0000", 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0200000000000001e-06 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423401+0000", 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015192100000000001 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423432+0000", 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1282999999999998e-05 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423439+0000", 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.241e-06 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423512+0000", 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3620000000000003e-05 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423529+0000", 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6813999999999999e-05 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424295+0000", 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076553700000000005 2024-09-11T08:44:01.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424352+0000", 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6918e-05 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.794332564999998, 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011035800000000001, 2024-09-11T08:44:01.235 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434677+0000", 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434683+0000", 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.212e-06 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434701+0000", 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8067000000000002e-05 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434721+0000", 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9562999999999999e-05 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434778+0000", 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7581000000000003e-05 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434800+0000", 2024-09-11T08:44:01.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2204e-05 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435738+0000", 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093753900000000004 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435780+0000", 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1683000000000001e-05 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.793869205999997, 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:44:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:44:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:44:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:44:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-11T08:44:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.793349417000002, 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010649730000000001, 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435660+0000", 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.51e-07 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435665+0000", 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435674+0000", 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435679+0000", 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6539999999999998e-06 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435815+0000", 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013680800000000001 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435838+0000", 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2821000000000001e-05 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436692+0000", 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085374500000000005 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436724+0000", 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2246000000000003e-05 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:01.249 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.792805043000001, 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:44:01.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'15)", 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.792311173000002, 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108423, 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436698+0000", 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4499999999999996e-07 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436737+0000", 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9226e-05 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436751+0000", 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3172e-05 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436758+0000", 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689999999999999e-06 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436816+0000", 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7506999999999997e-05 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436836+0000", 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0324000000000001e-05 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437746+0000", 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000910537 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437782+0000", 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5150000000000001e-05 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:38 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.787819126000002, 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010117800000000001, 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441190+0000", 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9999999999999996e-07 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441257+0000", 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6607999999999997e-05 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441269+0000", 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1953999999999999e-05 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441275+0000", 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9020000000000004e-06 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441315+0000", 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0506999999999997e-05 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441328+0000", 2024-09-11T08:44:01.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3165e-05 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442139+0000", 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081040300000000001 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442201+0000", 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2440999999999994e-05 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:39 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'25, mlcod=16'22)", 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.787384912, 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010054160000000001, 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0900000000000002e-07 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441721+0000", 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6898000000000006e-05 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441734+0000", 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.332e-05 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441744+0000", 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.465e-06 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441812+0000", 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7694999999999995e-05 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441834+0000", 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2614000000000001e-05 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442593+0000", 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075926599999999998 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442629+0000", 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5648999999999997e-05 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:53 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'31, mlcod=16'29)", 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.776009367, 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098036799999999991, 2024-09-11T08:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453000+0000", 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0400000000000005e-07 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453012+0000", 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1911e-05 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453024+0000", 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2257e-05 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453030+0000", 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0009999999999998e-06 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453076+0000", 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5516000000000003e-05 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453096+0000", 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0361e-05 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453934+0000", 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083800900000000002 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453980+0000", 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5509000000000001e-05 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.271 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.775437494000002, 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001121478, 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453572+0000", 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1600000000000001e-07 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453577+0000", 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4199999999999998e-06 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453588+0000", 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1134999999999999e-05 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453594+0000", 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453654+0000", 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9202e-05 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453668+0000", 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4375999999999999e-05 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454647+0000", 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097895199999999995 2024-09-11T08:44:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454693+0000", 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5558e-05 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-11T08:44:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.774884602999997, 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001004079, 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454125+0000", 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2399999999999997e-07 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454212+0000", 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6881999999999999e-05 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454234+0000", 2024-09-11T08:44:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2109000000000001e-05 2024-09-11T08:44:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454251+0000", 2024-09-11T08:44:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7552999999999999e-05 2024-09-11T08:44:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454294+0000", 2024-09-11T08:44:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2910000000000001e-05 2024-09-11T08:44:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454306+0000", 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1853e-05 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455087+0000", 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078128799999999999 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455128+0000", 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.066e-05 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.772947225999999, 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010907619999999999, 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456062+0000", 2024-09-11T08:44:01.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7899999999999998e-07 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456134+0000", 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1617000000000002e-05 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456145+0000", 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1581e-05 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456156+0000", 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0671e-05 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456200+0000", 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4159999999999997e-05 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456217+0000", 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7061000000000001e-05 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457093+0000", 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087593800000000004 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457152+0000", 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9055000000000003e-05 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.765081477000003, 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.764658883999999, 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001143493, 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.292 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2799999999999996e-07 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464395+0000", 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4858000000000003e-05 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464409+0000", 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3553000000000001e-05 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464416+0000", 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2899999999999997e-06 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464487+0000", 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0994000000000005e-05 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464520+0000", 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2790000000000003e-05 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465459+0000", 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093904100000000005 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465493+0000", 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4338999999999998e-05 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-11T08:44:01.296 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.752010745, 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001017771, 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7700000000000004e-07 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477043+0000", 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4397999999999997e-05 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477053+0000", 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0565e-05 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477063+0000", 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6169999999999996e-06 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477138+0000", 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5370000000000005e-05 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477151+0000", 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2962999999999999e-05 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477978+0000", 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082701499999999996 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.478016+0000", 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7166000000000003e-05 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:99 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'65, mlcod=16'63)", 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.604300174000002, 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010323559999999999, 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624715+0000", 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9739999999999996e-06 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624744+0000", 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8240000000000001e-05 2024-09-11T08:44:01.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624783+0000", 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9555000000000003e-05 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624838+0000", 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4728000000000002e-05 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624927+0000", 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9493000000000005e-05 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624955+0000", 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7966999999999998e-05 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625702+0000", 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074682700000000004 2024-09-11T08:44:01.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625741+0000", 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8572000000000003e-05 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:01.306 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.600883385000003, 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010216209999999999, 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:01.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628126+0000", 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1699999999999998e-07 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628133+0000", 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0639999999999996e-06 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628143+0000", 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628149+0000", 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628197+0000", 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8272000000000001e-05 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628217+0000", 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0171999999999999e-05 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629111+0000", 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089439299999999995 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629147+0000", 2024-09-11T08:44:01.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5317000000000001e-05 2024-09-11T08:44:01.311 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.311 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.311 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 49.488613698000002, 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001056343, 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:01.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7899999999999999e-07 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740399+0000", 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1389999999999997e-06 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740411+0000", 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1313e-05 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740415+0000", 2024-09-11T08:44:01.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1269999999999996e-06 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740462+0000", 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6659999999999997e-05 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740475+0000", 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3301e-05 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741430+0000", 2024-09-11T08:44:01.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095528300000000002 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741451+0000", 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1041000000000001e-05 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 49.488178544, 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099133399999999997, 2024-09-11T08:44:01.316 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740831+0000", 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740929+0000", 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8220000000000005e-05 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740943+0000", 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4121000000000001e-05 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740947+0000", 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3250000000000001e-06 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740990+0000", 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2842000000000001e-05 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741010+0000", 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0168000000000001e-05 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741802+0000", 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079172499999999996 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741821+0000", 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9434999999999999e-05 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.320 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 49.487082346999998, 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098299900000000003, 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741927+0000", 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3299999999999998e-07 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741932+0000", 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741938+0000", 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6099999999999997e-06 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741941+0000", 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.27e-06 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741971+0000", 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9708000000000001e-05 2024-09-11T08:44:01.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741982+0000", 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1148000000000001e-05 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742880+0000", 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089861100000000001 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742909+0000", 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8824999999999999e-05 2024-09-11T08:44:01.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.325 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:01.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:01.327 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:01.327+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:01.427 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:01.428+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:01.484 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:01.485 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:01.485 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:01.485 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:01.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:01.540 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:01.540+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:01.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T08:44:01.629 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:01.629+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:01.649 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:01.649 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:01.649 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:01.649 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:01.649 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:01.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:01.730 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:01.730+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:01.831 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:01.831+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:01.833 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:01.833 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:01.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:01.833 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 43.444529293000002, 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095305400000000005, 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7299999999999996e-07 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389734+0000", 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-05 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389772+0000", 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7768999999999997e-05 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389783+0000", 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839000000000001e-05 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389852+0000", 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9268000000000002e-05 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389880+0000", 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7597999999999999e-05 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390619+0000", 2024-09-11T08:44:01.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073945300000000005 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390671+0000", 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.440120026999999, 2024-09-11T08:44:01.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095448600000000005, 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:01.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9399999999999999e-07 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394137+0000", 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0331e-05 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394155+0000", 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7558999999999999e-05 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394170+0000", 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4591e-05 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394237+0000", 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7026999999999994e-05 2024-09-11T08:44:01.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394257+0000", 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0145000000000001e-05 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395041+0000", 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078400400000000004 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395081+0000", 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0635e-05 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(2.2 35/34 [PushOp(2:5cea7035:::rbd_info:head, version: 21'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@21'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.342016674, 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044003299999999999, 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:01.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492231+0000", 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9400000000000005e-07 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492245+0000", 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4115e-05 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492265+0000", 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9585e-05 2024-09-11T08:44:01.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492282+0000", 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7322000000000001e-05 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492387+0000", 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010562599999999999 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492396+0000", 2024-09-11T08:44:01.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4889999999999995e-06 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492670+0000", 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027430199999999998 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:01.847 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:01.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:01.932 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:01.933+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-11T08:44:02.020 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:02.020 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:02.020 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:02.020 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:02.033 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:02.033+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:02.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:02.133 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:02.134+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:02.187 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:02.187 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:02.188 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:02.188 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:02.188 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:02.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:02.243 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:02.244+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:02.311 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 44.928646125, 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025475459999999999, 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383750+0000", 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383766+0000", 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5752000000000001e-05 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383780+0000", 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4122e-05 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383881+0000", 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010105 2024-09-11T08:44:02.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383914+0000", 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2999e-05 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386246+0000", 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023319579999999999 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386291+0000", 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4435999999999999e-05 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(1.0 44/43 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.76798851899999998, 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001134822, 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544644+0000", 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00024222499999999999 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544669+0000", 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4910999999999999e-05 2024-09-11T08:44:02.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544703+0000", 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4557000000000001e-05 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544795+0000", 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1619999999999994e-05 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544805+0000", 2024-09-11T08:44:02.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0287e-05 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.545535+0000", 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073039999999999997 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(1.0 44/43 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:02.321 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.765063995, 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010332119999999999, 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:02.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547326+0000", 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6700000000000003e-07 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547537+0000", 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021147200000000001 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547561+0000", 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3702000000000001e-05 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547579+0000", 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8348000000000001e-05 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547690+0000", 2024-09-11T08:44:02.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110136 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547704+0000", 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.471e-05 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.548358+0000", 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065407699999999998 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4228.0:2 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.14007665899999999, 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00026821700000000001, 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4228", 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 2 2024-09-11T08:44:02.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:02.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172314+0000", 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7680000000000001e-06 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172322+0000", 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1070000000000005e-06 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172336+0000", 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3402e-05 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172348+0000", 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2666999999999999e-05 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172444+0000", 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5519999999999993e-05 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172581+0000", 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013675300000000001 2024-09-11T08:44:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:02.330 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:02.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:02.336 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:02.336+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:02.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:02.436 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:02.437+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:02.509 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:02.509 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:02.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:02.566 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:02.567+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:02.614 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:02.638 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:02.638+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:02.648 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:02.648 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:02.680 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:02.680 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:02.695 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:02.696 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:02.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:02.712 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:02.743 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:02.744 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:02.744 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:02.744+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:02.775 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:02.776 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:02.809 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:02.809 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:02.840 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:02.841 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:02.841+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:02.884 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:02.884 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:02.896 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:02.941 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:02.942+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:03.012 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:03.013 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:03.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:03.042 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:03.042+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:03.143 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:03.144+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:03.167 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:03.167 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:03.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:03.260 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:03.261+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:03.319 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:03.319 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:03.328 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:03.345 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:03.346+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:03.446 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:03.447+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:03.471 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:03.471 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:03.480 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:03.540 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:44:03.541 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:44:03.541 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-11T08:44:03.541 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:44:03.541 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:44:03.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-11T08:44:03.547 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:03.547+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:03.585 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:03.585 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:03.594 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:03.647 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:03.647 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T08:44:03.647 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:03.647 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:03.648+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:03.698 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:03.699 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:03.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:03.748 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:03.749+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:03.811 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:03.811 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:03.820 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:03.849 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:03.850+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:03.922 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:03.922 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:03.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:03.974 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:03.975+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:04.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T08:44:04.032 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:04.033 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:04.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:04.052 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:04.052+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:04.152 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:04.153+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:04.186 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:04.186 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:04.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:04.253 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:04.254+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:04.338 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:04.339 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:04.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:04.355 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:04.355+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:04.449 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:04.449 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:04.455 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:04.456+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:04.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-11T08:44:04.557 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:04.557+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:04.578 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:04.578 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:04.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:04.658 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:04.658+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:04.738 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:04.739 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:04.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:04.758 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:04.758+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:04.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:04.858 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:04.859+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:04.860 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:04.860 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:04.870 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:04.959 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:04.960+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:05.029 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:05.029 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:05.060 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:05.061+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:05.065 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:05.065 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:05.065 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:05.066 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:05.066 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:05.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:05.088 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:05.088 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:05.119 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:05.151 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:05.151 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:05.162 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:05.162+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:05.182 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:05.201 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:05.201 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:05.202 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:05.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:05.213 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:05.214 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:05.246 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:05.246 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:05.262 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:05.263+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:05.363 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:05.363+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:05.366 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:05.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.977512324000003, 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095305400000000005, 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7299999999999996e-07 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:05.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389734+0000", 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-05 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389772+0000", 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7768999999999997e-05 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389783+0000", 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839000000000001e-05 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389852+0000", 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9268000000000002e-05 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389880+0000", 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7597999999999999e-05 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390619+0000", 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073945300000000005 2024-09-11T08:44:05.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390671+0000", 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 45.973103058, 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095448600000000005, 2024-09-11T08:44:05.371 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9399999999999999e-07 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394137+0000", 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0331e-05 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394155+0000", 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7558999999999999e-05 2024-09-11T08:44:05.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394170+0000", 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4591e-05 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394237+0000", 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7026999999999994e-05 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:05.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394257+0000", 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0145000000000001e-05 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395041+0000", 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078400400000000004 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395081+0000", 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0635e-05 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(2.2 35/34 [PushOp(2:5cea7035:::rbd_info:head, version: 21'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@21'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.874999705, 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044003299999999999, 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492231+0000", 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9400000000000005e-07 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492245+0000", 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4115e-05 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492265+0000", 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9585e-05 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492282+0000", 2024-09-11T08:44:05.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7322000000000001e-05 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492387+0000", 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010562599999999999 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492396+0000", 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4889999999999995e-06 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:05.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492670+0000", 2024-09-11T08:44:05.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027430199999999998 2024-09-11T08:44:05.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.380 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.380 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.380 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:05.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:05.464 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:05.464+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:05.492 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:05.492 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:05.492 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:05.492 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:05.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:05.586 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:05.587+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:05.666 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:05.666+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:05.711 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:05.711 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:05.711 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:05.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:05.767 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:05.767+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:05.867 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:05.868+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:05.931 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:05.931 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:05.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:05.931 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:05.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 48.548210793000003, 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025475459999999999, 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:05.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383750+0000", 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383766+0000", 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5752000000000001e-05 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383780+0000", 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4122e-05 2024-09-11T08:44:05.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383881+0000", 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010105 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383914+0000", 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2999e-05 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:05.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386246+0000", 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023319579999999999 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386291+0000", 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4435999999999999e-05 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(1.0 44/43 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.387553187, 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001134822, 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544644+0000", 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00024222499999999999 2024-09-11T08:44:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544669+0000", 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4910999999999999e-05 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544703+0000", 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4557000000000001e-05 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544795+0000", 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1619999999999994e-05 2024-09-11T08:44:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544805+0000", 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0287e-05 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.545535+0000", 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073039999999999997 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(1.0 44/43 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.384628663, 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010332119999999999, 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:05.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547326+0000", 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6700000000000003e-07 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547537+0000", 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021147200000000001 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547561+0000", 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3702000000000001e-05 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547579+0000", 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8348000000000001e-05 2024-09-11T08:44:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547690+0000", 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110136 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547704+0000", 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.471e-05 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:05.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.548358+0000", 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065407699999999998 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4228.0:2 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.7596413269999998, 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00026821700000000001, 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:05.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4228", 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 2 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172314+0000", 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7680000000000001e-06 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172322+0000", 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1070000000000005e-06 2024-09-11T08:44:05.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172336+0000", 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3402e-05 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172348+0000", 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2666999999999999e-05 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:05.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172444+0000", 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5519999999999993e-05 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172581+0000", 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013675300000000001 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:05.949 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:05.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:05.991 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:05.992+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:06.070 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:06.070+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:06.102 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:06.102 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:06.102 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:06.102 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:06.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:06.170 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:06.171+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:06.235 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:06.236 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:06.236 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:06.236 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:06.236 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:06.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:06.272 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:06.272+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:06.372 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:06.372+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:06.457 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-11T08:44:06.458 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:06.470 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:06.473 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:06.474+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:06.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T08:44:06.585 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:06.585 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:06.585 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:06.585 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:06.594 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:06.609 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:06.610+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:06.675 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:06.675+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:06.698 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:06.698 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:06.698 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:06.699 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:06.699 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:06.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:06.791 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:06.792+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:06.852 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:06.852 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:06.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:06.852 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:44:06.852 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:06.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:06.876 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:06.876+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:06.977 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:06.977+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:07.004 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:07.004 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:07.005 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:07.005 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:07.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:07.078 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:07.078+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:07.116 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:07.116 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:07.116 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:07.116 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:07.117 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:07.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:07.179 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:07.179+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:07.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.889238345000003, 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:44:07.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.884052918000002, 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1896000000000003e-05, 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386389+0000", 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386394+0000", 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1839999999999998e-06 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386405+0000", 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386412+0000", 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9530000000000002e-06 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386453+0000", 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.074e-05 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:07.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386460+0000", 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4479999999999997e-06 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386470+0000", 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.281 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.878962936000001, 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000246512, 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 2 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391480+0000", 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1129999999999999e-06 2024-09-11T08:44:07.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391492+0000", 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1749999999999999e-05 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391522+0000", 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9886000000000002e-05 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391541+0000", 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9453000000000002e-05 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391624+0000", 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3307999999999996e-05 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391724+0000", 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100002 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.285 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.878469869, 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018776700000000001, 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:07.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391972+0000", 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391985+0000", 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3173000000000001e-05 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391995+0000", 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392003+0000", 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2449999999999998e-06 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392046+0000", 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2630999999999999e-05 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392159+0000", 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113069 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.289 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.882808605000001, 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:44:07.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:44:07.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:44:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:44:07.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:07.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:07.296 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.879670750999999, 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018276100000000001, 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390771+0000", 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4099999999999997e-07 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390778+0000", 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2230000000000001e-06 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390789+0000", 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.081e-05 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390795+0000", 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9070000000000004e-06 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390869+0000", 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4010000000000005e-05 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390876+0000", 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390953+0000", 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6971000000000006e-05 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:44:07.300 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 47.878559420000002, 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:44:07.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:07.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:44:07.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 47.875271757, 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104585, 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395170+0000", 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1009999999999999e-06 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395177+0000", 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395188+0000", 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0849999999999999e-05 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395194+0000", 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3999999999999997e-06 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395254+0000", 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0359000000000003e-05 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395261+0000", 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3679999999999998e-06 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395274+0000", 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.29e-05 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.311 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(2.2 35/34 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.777103899, 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000180452, 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:07.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3599999999999999e-07 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493342+0000", 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493350+0000", 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8990000000000001e-06 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493359+0000", 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8759999999999994e-06 2024-09-11T08:44:07.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493397+0000", 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8303999999999998e-05 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493517+0000", 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012013199999999999 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4228.0:1 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.0996566459999997, 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001851562, 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4228", 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170786+0000", 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9180000000000001e-06 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170796+0000", 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6649999999999998e-06 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170808+0000", 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1970999999999999e-05 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170818+0000", 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0682e-05 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170899+0000", 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0235999999999995e-05 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170904+0000", 2024-09-11T08:44:07.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.921e-06 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172541+0000", 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001637021 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172636+0000", 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5148000000000004e-05 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:07.321 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:07.321 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:07.322 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:07.322 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:07.322+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:07.422 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:07.423+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:07.464 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:07.467 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:07.467 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:07.467 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:07.467 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:07.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:07.501 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:07.530 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-11T08:44:07.560 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:07.561+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:07.562 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:07.594 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:07.621 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:07.621 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:07.624 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:07.625+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:07.625 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:07.625 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:07.629 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:07.655 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:07.656 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:07.686 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:07.725 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:07.726+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:07.773 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:07.773 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:07.782 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:07.826 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:07.827+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:07.925 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:07.925 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:07.927 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:07.928+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:07.933 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:08.028 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:08.029+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:08.076 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:08.076 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:08.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:08.170 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:08.171+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:08.229 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:08.230 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:08.230 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:08.230+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:08.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:08.331 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:08.331+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:08.395 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:08.395 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:08.408 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:08.432 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:08.433+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:08.517 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:08.517 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:08.527 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:08.533 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:08.534+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:08.634 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:08.634+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:08.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:08.657 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:44:08.657 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:44:08.657 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T08:44:08.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T08:44:08.734 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:08.735+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:08.813 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:08.813 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:08.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:08.835 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:08.836+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:08.936 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:08.936+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:08.941 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:08.941 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:08.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:09.037 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:09.037+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:09.102 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:09.102 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:09.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:09.138 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:09.138+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:09.221 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:09.221 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 48, 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "75755954-09e8-48d3-b9c1-83899efaff39", 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-11T08:42:57.181838+0000", 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-11T08:44:07.547946+0000", 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "last_up_change": "2024-09-11T08:43:09.343672+0000", 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "last_in_change": "2024-09-11T08:43:45.477548+0000", 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_num": 5804032, 2024-09-11T08:44:09.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_set": [ 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "nodeep-scrub", 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "pglog_hardlimit", 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "purged_snapdirs", 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_deletes", 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "sortbitwise" 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_version": 5, 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T08:44:09.231 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_snapshot": "", 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_max": 2, 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd": 8, 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "require_min_compat_client": "luminous", 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "min_compat_client": "luminous", 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "require_osd_release": "squid", 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_crimson": false, 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "pools": [ 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 1, 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": ".mgr", 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-11T08:43:08.201248+0000", 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool", 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-11T08:44:09.232 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 1, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 1, 2024-09-11T08:44:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 1, 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 1, 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 1, 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-11T08:44:09.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "17", 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 0, 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 0, 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-11T08:44:09.235 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-11T08:44:09.236 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:44:09.237 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "options": { 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_max": 32, 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_min": 1 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": {} 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-11T08:44:09.238 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 8, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 8, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.25, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 2, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 2, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 2, 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": "rbd", 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-11T08:43:16.189979+0000", 2024-09-11T08:44:09.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 8193, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:44:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 8, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 8, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 8, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 8, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 8, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-11T08:44:09.241 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "21", 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 2, 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 21, 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-11T08:44:09.242 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-11T08:44:09.243 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:44:09.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "options": {}, 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd": {} 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-11T08:44:09.245 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 1.9900000095367432, 2024-09-11T08:44:09.246 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 1.9900000095367432, 2024-09-11T08:44:09.247 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.87999999523162842, 2024-09-11T08:44:09.247 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 1.75, 2024-09-11T08:44:09.247 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 1.75, 2024-09-11T08:44:09.247 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-11T08:44:09.247 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-11T08:44:09.247 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-11T08:44:09.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "ac6b86cf-4ee8-49eb-a680-a350b1c9879c", 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:09.248 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 44, 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6808", 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6809", 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6810", 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6811", 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6814", 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6815", 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.252 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6812", 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6813", 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6809/2540486849", 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6811/2540486849", 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6815/2540486849", 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6813/2540486849", 2024-09-11T08:44:09.253 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "e27d776b-f77b-4111-a536-5f5d8030c696", 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:09.254 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 34, 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6824", 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.255 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6825", 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6826", 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.256 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6827", 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6830", 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.257 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6831", 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6828", 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6829", 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6825/1040844445", 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6827/1040844445", 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6831/1040844445", 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6829/1040844445", 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "6b8e44f6-395b-447d-a5d3-c19a0a4be61b", 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:09.260 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 45, 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6816", 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6817", 2024-09-11T08:44:09.261 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6818", 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.263 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6819", 2024-09-11T08:44:09.263 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:09.263 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.263 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.263 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:09.263 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6822", 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6823", 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6820", 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6821", 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.265 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6817/1231172466", 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6819/1231172466", 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6823/1231172466", 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6821/1231172466", 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "1d1e1983-4c89-4cb4-a6b4-af244f941a7a", 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:09.266 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 13, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 0, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-09-11T08:44:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6802", 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6803", 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:09.269 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6806", 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6807", 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6804", 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6805", 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6801/226138367", 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6803/226138367", 2024-09-11T08:44:09.271 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6807/226138367", 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6805/226138367", 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "fd9239ee-3a74-4399-af2f-1e9a63a2f642", 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:09.272 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 13, 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 0, 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6800", 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:09.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6801", 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6802", 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:09.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6803", 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.275 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6806", 2024-09-11T08:44:09.276 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:09.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.276 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6807", 2024-09-11T08:44:09.276 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6804", 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6805", 2024-09-11T08:44:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6801/374760845", 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6803/374760845", 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6807/374760845", 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6805/374760845", 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:09.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "cfff2555-5ad4-4629-a725-afadf9b6a96e", 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:09.279 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 34, 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6824", 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6825", 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:09.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6826", 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6827", 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.282 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6830", 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6831", 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6828", 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6829", 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.284 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6825/498386186", 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6827/498386186", 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6831/498386186", 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6829/498386186", 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-11T08:44:09.285 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "4588b13e-af90-48a7-a10d-f7a185a902e2", 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 22, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6808", 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6809", 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6810", 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6811", 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.288 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6814", 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6815", 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:09.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6812", 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6813", 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6809/1261869816", 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6811/1261869816", 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6815/1261869816", 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6813/1261869816", 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.292 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-11T08:44:09.292 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "12da1d21-9c59-4462-9d5a-61801dab3e4d", 2024-09-11T08:44:09.292 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:09.292 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:09.292 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:09.292 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 35, 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6816", 2024-09-11T08:44:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6817", 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6818", 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:09.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6819", 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:09.295 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6822", 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6823", 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.296 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6820", 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6821", 2024-09-11T08:44:09.297 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6817/2975777507", 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6819/2975777507", 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2975777507", 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2975777507", 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:09.298 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_xinfo": [ 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.394370+0000", 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:09.299 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.813228+0000", 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:09.300 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.425286+0000", 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.147591+0000", 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:09.302 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:07.119248+0000", 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.653838+0000", 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:09.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.459308+0000", 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.492589+0000", 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:09.305 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap": [ 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: "pgid": "1.0", 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: 0 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_items": [], 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_primaries": [], 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_temp": [], 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_temp": [], 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklist": {}, 2024-09-11T08:44:09.306 INFO:teuthology.orchestra.run.smithi045.stdout: "range_blocklist": {}, 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profiles": { 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default": { 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-failure-domain": "osd", 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "k": "2", 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "m": "1", 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin": "jerasure", 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "technique": "reed_sol_van" 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps_queue": [], 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "new_removed_snaps": [], 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "new_purged_snaps": [], 2024-09-11T08:44:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_node_flags": {}, 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout: "device_class_flags": {}, 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": { 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_enabled": false, 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_bucket_count": 0, 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout: "degraded_stretch_mode": 0, 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout: "recovering_stretch_mode": 0, 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_bucket": 0 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:09.308 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:09.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:09.312 INFO:tasks.thrashosds.thrasher:j is {'epoch': 48, 'fsid': '75755954-09e8-48d3-b9c1-83899efaff39', 'created': '2024-09-11T08:42:57.181838+0000', 'modified': '2024-09-11T08:44:07.547946+0000', 'last_up_change': '2024-09-11T08:43:09.343672+0000', 'last_in_change': '2024-09-11T08:43:45.477548+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:43:08.201248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T08:43:16.189979+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ac6b86cf-4ee8-49eb-a680-a350b1c9879c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6808', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6809', 'nonce': 2540486849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6810', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6811', 'nonce': 2540486849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6814', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6815', 'nonce': 2540486849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6812', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6813', 'nonce': 2540486849}]}, 'public_addr': '172.21.15.45:6809/2540486849', 'cluster_addr': '172.21.15.45:6811/2540486849', 'heartbeat_back_addr': '172.21.15.45:6815/2540486849', 'heartbeat_front_addr': '172.21.15.45:6813/2540486849', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e27d776b-f77b-4111-a536-5f5d8030c696', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6824', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6825', 'nonce': 1040844445}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6826', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6827', 'nonce': 1040844445}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6830', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6831', 'nonce': 1040844445}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6828', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6829', 'nonce': 1040844445}]}, 'public_addr': '172.21.15.45:6825/1040844445', 'cluster_addr': '172.21.15.45:6827/1040844445', 'heartbeat_back_addr': '172.21.15.45:6831/1040844445', 'heartbeat_front_addr': '172.21.15.45:6829/1040844445', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '6b8e44f6-395b-447d-a5d3-c19a0a4be61b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6816', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6817', 'nonce': 1231172466}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6818', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6819', 'nonce': 1231172466}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6822', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6823', 'nonce': 1231172466}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6820', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6821', 'nonce': 1231172466}]}, 'public_addr': '172.21.15.45:6817/1231172466', 'cluster_addr': '172.21.15.45:6819/1231172466', 'heartbeat_back_addr': '172.21.15.45:6823/1231172466', 'heartbeat_front_addr': '172.21.15.45:6821/1231172466', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1d1e1983-4c89-4cb4-a6b4-af244f941a7a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6800', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6801', 'nonce': 226138367}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6802', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6803', 'nonce': 226138367}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6806', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6807', 'nonce': 226138367}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6804', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6805', 'nonce': 226138367}]}, 'public_addr': '172.21.15.45:6801/226138367', 'cluster_addr': '172.21.15.45:6803/226138367', 'heartbeat_back_addr': '172.21.15.45:6807/226138367', 'heartbeat_front_addr': '172.21.15.45:6805/226138367', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd9239ee-3a74-4399-af2f-1e9a63a2f642', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6800', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6801', 'nonce': 374760845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6802', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6803', 'nonce': 374760845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6806', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6807', 'nonce': 374760845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6804', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6805', 'nonce': 374760845}]}, 'public_addr': '172.21.15.86:6801/374760845', 'cluster_addr': '172.21.15.86:6803/374760845', 'heartbeat_back_addr': '172.21.15.86:6807/374760845', 'heartbeat_front_addr': '172.21.15.86:6805/374760845', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'cfff2555-5ad4-4629-a725-afadf9b6a96e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6824', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6825', 'nonce': 498386186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6826', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6827', 'nonce': 498386186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6830', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6831', 'nonce': 498386186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6828', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6829', 'nonce': 498386186}]}, 'public_addr': '172.21.15.86:6825/498386186', 'cluster_addr': '172.21.15.86:6827/498386186', 'heartbeat_back_addr': '172.21.15.86:6831/498386186', 'heartbeat_front_addr': '172.21.15.86:6829/498386186', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '4588b13e-af90-48a7-a10d-f7a185a902e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6808', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6809', 'nonce': 1261869816}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6810', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6811', 'nonce': 1261869816}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6814', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6815', 'nonce': 1261869816}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6812', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6813', 'nonce': 1261869816}]}, 'public_addr': '172.21.15.86:6809/1261869816', 'cluster_addr': '172.21.15.86:6811/1261869816', 'heartbeat_back_addr': '172.21.15.86:6815/1261869816', 'heartbeat_front_addr': '172.21.15.86:6813/1261869816', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '12da1d21-9c59-4462-9d5a-61801dab3e4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6816', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6817', 'nonce': 2975777507}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6818', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6819', 'nonce': 2975777507}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6822', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6823', 'nonce': 2975777507}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6820', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6821', 'nonce': 2975777507}]}, 'public_addr': '172.21.15.86:6817/2975777507', 'cluster_addr': '172.21.15.86:6819/2975777507', 'heartbeat_back_addr': '172.21.15.86:6823/2975777507', 'heartbeat_front_addr': '172.21.15.86:6821/2975777507', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.394370+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.813228+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.425286+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.147591+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:07.119248+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.653838+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.459308+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.492589+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.0', 'osds': [2, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T08:44:09.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:44:09.313 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:09.310+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:09.410 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:09.411+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:09.511 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:09.512+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:09.522 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:09.522 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:09.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:09.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T08:44:09.621 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:09.621+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:09.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:09.749 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:09.749 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:09.750 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:09.750+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:09.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:09.852 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:09.852+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:09.886 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:09.886 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:44:09.914 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-11T08:44:08.159623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":242300,"kb_used_data":5260,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732420,"statfs":{"total":773094113280,"available":772845998080,"internally_reserved":0,"allocated":5386240,"data_stored":2867745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001020"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T08:44:03.102700+0000","last_change":"2024-09-11T08:43:19.626333+0000","last_active":"2024-09-11T08:44:03.102700+0000","last_peered":"2024-09-11T08:44:03.102700+0000","last_clean":"2024-09-11T08:44:03.102700+0000","last_became_active":"2024-09-11T08:43:17.384965+0000","last_became_peered":"2024-09-11T08:43:17.384965+0000","last_unstale":"2024-09-11T08:44:03.102700+0000","last_undegraded":"2024-09-11T08:44:03.102700+0000","last_fullsized":"2024-09-11T08:44:03.102700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:09.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037094500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T08:44:02.983403+0000","last_change":"2024-09-11T08:43:46.492417+0000","last_active":"2024-09-11T08:44:02.983403+0000","last_peered":"2024-09-11T08:44:02.983403+0000","last_clean":"2024-09-11T08:44:02.983403+0000","last_became_active":"2024-09-11T08:43:46.492204+0000","last_became_peered":"2024-09-11T08:43:46.492204+0000","last_unstale":"2024-09-11T08:44:02.983403+0000","last_undegraded":"2024-09-11T08:44:02.983403+0000","last_fullsized":"2024-09-11T08:44:02.983403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:43.614795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T08:44:04.546967+0000","last_change":"2024-09-11T08:43:19.626282+0000","last_active":"2024-09-11T08:44:04.546967+0000","last_peered":"2024-09-11T08:44:04.546967+0000","last_clean":"2024-09-11T08:44:04.546967+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:44:04.546967+0000","last_undegraded":"2024-09-11T08:44:04.546967+0000","last_fullsized":"2024-09-11T08:44:04.546967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:10.938383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049065999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T08:44:02.983355+0000","last_change":"2024-09-11T08:43:46.864029+0000","last_active":"2024-09-11T08:44:02.983355+0000","last_peered":"2024-09-11T08:44:02.983355+0000","last_clean":"2024-09-11T08:44:02.983355+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:44:02.983355+0000","last_undegraded":"2024-09-11T08:44:02.983355+0000","last_fullsized":"2024-09-11T08:44:02.983355+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:51.854546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":86,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T08:44:03.102701+0000","last_change":"2024-09-11T08:43:46.493691+0000","last_active":"2024-09-11T08:44:03.102701+0000","last_peered":"2024-09-11T08:44:03.102701+0000","last_clean":"2024-09-11T08:44:03.102701+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:44:03.102701+0000","last_undegraded":"2024-09-11T08:44:03.102701+0000","last_fullsized":"2024-09-11T08:44:03.102701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:14.454230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010925100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T08:44:04.549895+0000","last_change":"2024-09-11T08:43:19.469399+0000","last_active":"2024-09-11T08:44:04.549895+0000","last_peered":"2024-09-11T08:44:04.549895+0000","last_clean":"2024-09-11T08:44:04.549895+0000","last_became_active":"2024-09-11T08:43:17.383480+0000","last_became_peered":"2024-09-11T08:43:17.383480+0000","last_unstale":"2024-09-11T08:44:04.549895+0000","last_undegraded":"2024-09-11T08:44:04.549895+0000","last_fullsized":"2024-09-11T08:44:04.549895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:43.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040896800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T08:44:04.546967+0000","last_change":"2024-09-11T08:43:47.895970+0000","last_active":"2024-09-11T08:44:04.546967+0000","last_peered":"2024-09-11T08:44:04.546967+0000","last_clean":"2024-09-11T08:44:04.546967+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:44:04.546967+0000","last_undegraded":"2024-09-11T08:44:04.546967+0000","last_fullsized":"2024-09-11T08:44:04.546967+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:23:07.574724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048316400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T08:44:03.102750+0000","last_change":"2024-09-11T08:43:19.394034+0000","last_active":"2024-09-11T08:44:03.102750+0000","last_peered":"2024-09-11T08:44:03.102750+0000","last_clean":"2024-09-11T08:44:03.102750+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:44:03.102750+0000","last_undegraded":"2024-09-11T08:44:03.102750+0000","last_fullsized":"2024-09-11T08:44:03.102750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:54.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719799999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T08:44:04.550104+0000","last_change":"2024-09-11T08:44:02.988511+0000","last_active":"2024-09-11T08:44:04.550104+0000","last_peered":"2024-09-11T08:44:04.550104+0000","last_clean":"2024-09-11T08:44:04.550104+0000","last_became_active":"2024-09-11T08:44:02.988115+0000","last_became_peered":"2024-09-11T08:44:02.988115+0000","last_unstale":"2024-09-11T08:44:04.550104+0000","last_undegraded":"2024-09-11T08:44:04.550104+0000","last_fullsized":"2024-09-11T08:44:04.550104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:51.312057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":212923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":207452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509453,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":520192,"data_stored":207471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1122304,"data_stored":803291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":520192,"data_stored":207471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":803291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":212923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35864,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335976,"statfs":{"total":96636764160,"available":96600039424,"internally_reserved":0,"allocated":528384,"data_stored":212923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:44:09.915 INFO:tasks.thrashosds.thrasher:Setting 2.1 to [0, 1] 2024-09-11T08:44:09.915 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.1', '0', '1'] 2024-09-11T08:44:09.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.1 0 1 2024-09-11T08:44:09.916 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:09.915+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:09.917 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:09.952 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:09.980 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:09.980 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:09.985 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:09.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:10.015 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:10.015+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:10.017 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:10.018 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:10.050 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:10.051 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:10.084 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:10.116 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:10.117+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:10.119 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:10.119 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:10.152 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:10.182 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:10.182 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:10.182 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:10.182 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:10.183 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:10.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:10.217 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:10.218+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:10.318 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:10.319+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:10.343 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-11T08:44:10.344 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:10.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:10.420 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:10.420+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:10.469 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:10.469 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:10.469 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:10.469 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:10.478 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:10.520 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:10.520+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-11T08:44:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:set 2.1 pg_upmap mapping to [0,1] 2024-09-11T08:44:10.621 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:10.622+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:10.622 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:10.623 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:10.623 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:10.623 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:10.623 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:10.632 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:10.722 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:10.722+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:10.776 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:10.776 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:10.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:10.777 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:44:10.777 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:10.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:10.822 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:10.823+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:10.923 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:10.924+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:10.929 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:10.929 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:10.929 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:10.929 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:10.937 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:11.024 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:11.025+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:11.080 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:11.080 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:11.080 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:11.080 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:11.080 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:11.089 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:11.125 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:11.125+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:11.193 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:11.193 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:11.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:11.193 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.812657489999999, 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:44:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.807472062999999, 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1896000000000003e-05, 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386389+0000", 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386394+0000", 2024-09-11T08:44:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1839999999999998e-06 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386405+0000", 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386412+0000", 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9530000000000002e-06 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386453+0000", 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.074e-05 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386460+0000", 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4479999999999997e-06 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386470+0000", 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.802382080999998, 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000246512, 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 2 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391480+0000", 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1129999999999999e-06 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391492+0000", 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1749999999999999e-05 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391522+0000", 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9886000000000002e-05 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391541+0000", 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9453000000000002e-05 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391624+0000", 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3307999999999996e-05 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391724+0000", 2024-09-11T08:44:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100002 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.801889013999997, 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018776700000000001, 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391972+0000", 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391985+0000", 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3173000000000001e-05 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391995+0000", 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392003+0000", 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2449999999999998e-06 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392046+0000", 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2630999999999999e-05 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392159+0000", 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113069 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.806227749999998, 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:44:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:44:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.803089896000003, 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018276100000000001, 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390771+0000", 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4099999999999997e-07 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390778+0000", 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2230000000000001e-06 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390789+0000", 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.081e-05 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390795+0000", 2024-09-11T08:44:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9070000000000004e-06 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390869+0000", 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4010000000000005e-05 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390876+0000", 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390953+0000", 2024-09-11T08:44:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6971000000000006e-05 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 51.801978564999999, 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:44:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:44:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 51.798690901999997, 2024-09-11T08:44:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104585, 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395170+0000", 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1009999999999999e-06 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395177+0000", 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395188+0000", 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0849999999999999e-05 2024-09-11T08:44:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395194+0000", 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3999999999999997e-06 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395254+0000", 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0359000000000003e-05 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395261+0000", 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3679999999999998e-06 2024-09-11T08:44:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395274+0000", 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.29e-05 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(2.2 35/34 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.700523044000001, 2024-09-11T08:44:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000180452, 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3599999999999999e-07 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493342+0000", 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493350+0000", 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8990000000000001e-06 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493359+0000", 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8759999999999994e-06 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493397+0000", 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8303999999999998e-05 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493517+0000", 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012013199999999999 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4228.0:1 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0230757910000001, 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001851562, 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4228", 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170786+0000", 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9180000000000001e-06 2024-09-11T08:44:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170796+0000", 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6649999999999998e-06 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170808+0000", 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1970999999999999e-05 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170818+0000", 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0682e-05 2024-09-11T08:44:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170899+0000", 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0235999999999995e-05 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170904+0000", 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.921e-06 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172541+0000", 2024-09-11T08:44:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001637021 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172636+0000", 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5148000000000004e-05 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:11.242 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:11.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:11.244 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:11.244+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:11.344 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:11.345+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:11.388 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:11.389 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:11.389 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:11.389 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:11.396 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:11.445 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:11.446+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:11.499 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:11.500 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:11.500 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:11.500 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:11.500 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:11.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:11.546 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:11.546+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:11.647 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:11.648+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:11.652 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:11.652 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:11.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:11.652 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:44:11.652 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:11.661 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:11.748 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:11.749+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:11.805 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:11.805 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:11.805 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:11.805 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:11.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:11.855 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:11.855+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:11.950 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:11.951+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:11.956 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:11.956 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:11.956 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:11.956 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:11.957 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:11.965 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:12.051 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:12.052+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 62.751711769000003, 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:44:12.113 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.120 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.120 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:12.120 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.677609406999998, 2024-09-11T08:44:12.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:44:12.120 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:44:12.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.126 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.677537919000002, 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.127 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:44:12.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:44:12.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:44:12.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:44:12.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:44:12.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.133 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.677514307999999, 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:44:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:44:12.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.139 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.635746924999999, 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.635723059, 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:44:12.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.152 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.635617001, 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:44:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.158 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.483914525000003, 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:44:12.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:44:12.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.483885428999997, 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:44:12.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.170 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.483859090000003, 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:44:12.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:44:12.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:44:12.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.483789791, 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.177 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:44:12.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:44:12.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.483764839000003, 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:44:12.183 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:44:12.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:44:12.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.074067702999997, 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:44:12.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.073980089999999, 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:12.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:44:12.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.201 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.073940581999999, 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:44:12.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.207 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.073914377000001, 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:12.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:44:12.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:44:12.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.217 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.073863121999999, 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:44:12.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.223 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.073851202999997, 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:44:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.073837326000003, 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:44:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.073795783999998, 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:44:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:12.242 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:12.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:12.243 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:12.243+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:12.244 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:12.344 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:12.345+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:12.366 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:12.366 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:12.366 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:12.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:12.408 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:12.409 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:12.442 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:12.444 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:12.445+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:12.477 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:12.510 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:12.511 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:12.544 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:12.544 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:12.545 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:12.546+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:12.571 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:12.571 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:12.578 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:12.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:12.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T08:44:12.610 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:12.637 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:12.654 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:12.655+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:12.748 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:12.748+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:12.788 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:12.788 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:12.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:12.885 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:12.886+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:12.949 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:12.949+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:12.961 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:12.961 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:12.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:13.054 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:13.055+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:13.141 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:13.141 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:13.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:13.151 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:13.151+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:13.252 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:13.252+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:13.352 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:13.353+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:13.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:13.457 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:13.458+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:13.555 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:13.555+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-11T08:44:13.583 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:13.583 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:13.597 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:13.656 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:13.657+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:13.757 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:13.757+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:13.804 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:13.804 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:13.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:13.858 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:13.858+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:13.959 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:13.959+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:13.967 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:13.967 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:13.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:14.059 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:14.060+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:14.087 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:14.087 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:14.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:14.161 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:14.162+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:14.233 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:14.233 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:14.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:14.262 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:14.263+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:14.364 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:14.364+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:14.396 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:14.396 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:14.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:14.465 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:14.466+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:14.520 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:14.520 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:14.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:14.566 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:14.567+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:14.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:14.672 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:14.672 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:14.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:14.699 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:14.700+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:14.770 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:14.770+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:14.784 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:14.784 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:14.785 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:14.785 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:14.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:14.801 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:14.801 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:14.819 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:14.819 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:14.836 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:14.853 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:14.872 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:14.873 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:14.873+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:14.887 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:14.903 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:14.904 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:14.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:14.974 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:14.974+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:15.048 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:15.048 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:15.057 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:15.100 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:15.101+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:15.161 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 57.780867397999998, 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:44:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:44:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 57.775681970999997, 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1896000000000003e-05, 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386389+0000", 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386394+0000", 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1839999999999998e-06 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386405+0000", 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386412+0000", 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9530000000000002e-06 2024-09-11T08:44:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386453+0000", 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.074e-05 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386460+0000", 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4479999999999997e-06 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386470+0000", 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 57.770591989000003, 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000246512, 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 2 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391480+0000", 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1129999999999999e-06 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391492+0000", 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1749999999999999e-05 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391522+0000", 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9886000000000002e-05 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391541+0000", 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9453000000000002e-05 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391624+0000", 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3307999999999996e-05 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391724+0000", 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100002 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 57.770098922000003, 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018776700000000001, 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391972+0000", 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391985+0000", 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3173000000000001e-05 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391995+0000", 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392003+0000", 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2449999999999998e-06 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392046+0000", 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2630999999999999e-05 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392159+0000", 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113069 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.774437657999997, 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:44:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:44:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.771299804000002, 2024-09-11T08:44:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018276100000000001, 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390771+0000", 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4099999999999997e-07 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390778+0000", 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2230000000000001e-06 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390789+0000", 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.081e-05 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390795+0000", 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9070000000000004e-06 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390869+0000", 2024-09-11T08:44:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4010000000000005e-05 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390876+0000", 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390953+0000", 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6971000000000006e-05 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.770188472999997, 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:44:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:44:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.766900810000003, 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104585, 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395170+0000", 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1009999999999999e-06 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395177+0000", 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395188+0000", 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0849999999999999e-05 2024-09-11T08:44:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395194+0000", 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3999999999999997e-06 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395254+0000", 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0359000000000003e-05 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395261+0000", 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3679999999999998e-06 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395274+0000", 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.29e-05 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(2.2 35/34 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 28.668732951999999, 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000180452, 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3599999999999999e-07 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493342+0000", 2024-09-11T08:44:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493350+0000", 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8990000000000001e-06 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493359+0000", 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8759999999999994e-06 2024-09-11T08:44:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493397+0000", 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8303999999999998e-05 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493517+0000", 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012013199999999999 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4228.0:1 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.991285699000001, 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001851562, 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4228", 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170786+0000", 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9180000000000001e-06 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170796+0000", 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6649999999999998e-06 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170808+0000", 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1970999999999999e-05 2024-09-11T08:44:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170818+0000", 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0682e-05 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170899+0000", 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0235999999999995e-05 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170904+0000", 2024-09-11T08:44:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.921e-06 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172541+0000", 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001637021 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172636+0000", 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5148000000000004e-05 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:15.215 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:15.215 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:15.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:15.217 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:15.216+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:15.316 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:15.317+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:15.361 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:15.361 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:15.361 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:15.361 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:15.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:15.417 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:15.417+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:15.472 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:15.472 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:15.472 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:15.472 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:15.473 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:15.480 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:15.518 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:15.518+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:15.586 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:44:15.587 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:44:15.587 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-11T08:44:15.587 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:44:15.587 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:44:15.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-11T08:44:15.597 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:15.635 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:15.641 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:15.642+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:15.691 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T08:44:15.692 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:15.741 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:15.742+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:15.779 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:15.779 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:15.779 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:15.779 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:15.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:15.821 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:15.821+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:15.922 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:15.922+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:15.931 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:15.931 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:15.931 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:15.931 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:15.931 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:15.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:16.023 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:16.023+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:16.046 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:16.046 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:16.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:16.046 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:16.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.046 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.685284972000005, 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:16.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.048 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:16.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:44:16.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:44:16.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:44:16.054 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.054 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.054 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.055 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.055 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:16.055 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.61118261, 2024-09-11T08:44:16.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:44:16.056 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.056 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.056 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.056 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.056 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:44:16.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.057 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:16.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:16.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:16.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:44:16.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:44:16.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:44:16.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:44:16.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:44:16.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:44:16.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:44:16.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:44:16.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:44:16.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:44:16.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:44:16.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:44:16.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:44:16.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:44:16.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.611111121999997, 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:44:16.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:44:16.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.611087510999994, 2024-09-11T08:44:16.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:16.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.074 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.569320128000001, 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:44:16.077 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:44:16.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:44:16.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.569296261999995, 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.084 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:16.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.569190203999995, 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:44:16.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.417487727999998, 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.097 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:44:16.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.417458632000006, 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:16.111 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.417432293000005, 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:16.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.417362994000001, 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:44:16.117 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.417338041999997, 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.007640906000006, 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:44:16.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:44:16.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:44:16.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:44:16.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.134 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.007553293000001, 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:16.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.140 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.007513785, 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.141 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:44:16.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.007487580000003, 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:44:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:44:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.007436325, 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:44:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:44:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.007424405999998, 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:44:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:44:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:44:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:44:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.007410528999998, 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:44:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.007368987000007, 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:44:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:44:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:44:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:16.178 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:16.180 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:16.180 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:16.180+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:16.281 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:16.281+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:16.296 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:16.297 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:16.297 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:16.297 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:16.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:16.382 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:16.382+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:16.418 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:16.418 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:16.418 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:16.418 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:16.418 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:16.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:16.483 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:16.484+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:16.538 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:16.538 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:16.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:16.538 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:16.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.538 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-11T08:44:16.538 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.113711455000001, 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011043540000000001, 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423249+0000", 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0200000000000001e-06 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423401+0000", 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015192100000000001 2024-09-11T08:44:16.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423432+0000", 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1282999999999998e-05 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423439+0000", 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.241e-06 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423512+0000", 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3620000000000003e-05 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423529+0000", 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6813999999999999e-05 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424295+0000", 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076553700000000005 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424352+0000", 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6918e-05 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.102282672000001, 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011035800000000001, 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434677+0000", 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-11T08:44:16.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434683+0000", 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.212e-06 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434701+0000", 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8067000000000002e-05 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434721+0000", 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9562999999999999e-05 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434778+0000", 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7581000000000003e-05 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434800+0000", 2024-09-11T08:44:16.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2204e-05 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435738+0000", 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093753900000000004 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435780+0000", 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1683000000000001e-05 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.548 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.101819312999993, 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.101299523999998, 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010649730000000001, 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435660+0000", 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.51e-07 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435665+0000", 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435674+0000", 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435679+0000", 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6539999999999998e-06 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435815+0000", 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013680800000000001 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435838+0000", 2024-09-11T08:44:16.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2821000000000001e-05 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436692+0000", 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085374500000000005 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436724+0000", 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2246000000000003e-05 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.100755149999998, 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:44:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'15)", 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.100261279999998, 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108423, 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436698+0000", 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4499999999999996e-07 2024-09-11T08:44:16.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436737+0000", 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9226e-05 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436751+0000", 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3172e-05 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436758+0000", 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689999999999999e-06 2024-09-11T08:44:16.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436816+0000", 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7506999999999997e-05 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436836+0000", 2024-09-11T08:44:16.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0324000000000001e-05 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437746+0000", 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000910537 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437782+0000", 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5150000000000001e-05 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:38 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.095769232999999, 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010117800000000001, 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441190+0000", 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9999999999999996e-07 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441257+0000", 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6607999999999997e-05 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441269+0000", 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1953999999999999e-05 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441275+0000", 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9020000000000004e-06 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441315+0000", 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0506999999999997e-05 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441328+0000", 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3165e-05 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442139+0000", 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081040300000000001 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442201+0000", 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2440999999999994e-05 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:39 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'25, mlcod=16'22)", 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.095335019000004, 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010054160000000001, 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0900000000000002e-07 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441721+0000", 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6898000000000006e-05 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441734+0000", 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.332e-05 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441744+0000", 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.465e-06 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441812+0000", 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7694999999999995e-05 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441834+0000", 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2614000000000001e-05 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442593+0000", 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075926599999999998 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442629+0000", 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5648999999999997e-05 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:53 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'31, mlcod=16'29)", 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:16.576 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.083959473999997, 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098036799999999991, 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453000+0000", 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0400000000000005e-07 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453012+0000", 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1911e-05 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453024+0000", 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2257e-05 2024-09-11T08:44:16.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453030+0000", 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0009999999999998e-06 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453076+0000", 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5516000000000003e-05 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453096+0000", 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0361e-05 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453934+0000", 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083800900000000002 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453980+0000", 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5509000000000001e-05 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.083387600999998, 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001121478, 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453572+0000", 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1600000000000001e-07 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453577+0000", 2024-09-11T08:44:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4199999999999998e-06 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453588+0000", 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1134999999999999e-05 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453594+0000", 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453654+0000", 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9202e-05 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453668+0000", 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4375999999999999e-05 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454647+0000", 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097895199999999995 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454693+0000", 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5558e-05 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.08283471, 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001004079, 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454125+0000", 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2399999999999997e-07 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454212+0000", 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6881999999999999e-05 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454234+0000", 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2109000000000001e-05 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454251+0000", 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7552999999999999e-05 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454294+0000", 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2910000000000001e-05 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454306+0000", 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1853e-05 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455087+0000", 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078128799999999999 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455128+0000", 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.066e-05 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.080897332999996, 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010907619999999999, 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456062+0000", 2024-09-11T08:44:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7899999999999998e-07 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456134+0000", 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1617000000000002e-05 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456145+0000", 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1581e-05 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456156+0000", 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0671e-05 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456200+0000", 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4159999999999997e-05 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456217+0000", 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7061000000000001e-05 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457093+0000", 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087593800000000004 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457152+0000", 2024-09-11T08:44:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9055000000000003e-05 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.073031584000006, 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.596 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:44:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-11T08:44:16.599 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.072608990999996, 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001143493, 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2799999999999996e-07 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464395+0000", 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4858000000000003e-05 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464409+0000", 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3553000000000001e-05 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464416+0000", 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2899999999999997e-06 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464487+0000", 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0994000000000005e-05 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464520+0000", 2024-09-11T08:44:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2790000000000003e-05 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465459+0000", 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093904100000000005 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465493+0000", 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4338999999999998e-05 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.059960852000003, 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001017771, 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7700000000000004e-07 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477043+0000", 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4397999999999997e-05 2024-09-11T08:44:16.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477053+0000", 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0565e-05 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477063+0000", 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6169999999999996e-06 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477138+0000", 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5370000000000005e-05 2024-09-11T08:44:16.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477151+0000", 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2962999999999999e-05 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477978+0000", 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082701499999999996 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.478016+0000", 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7166000000000003e-05 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.607 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:99 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'65, mlcod=16'63)", 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.912250280999999, 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010323559999999999, 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:16.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624715+0000", 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9739999999999996e-06 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624744+0000", 2024-09-11T08:44:16.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8240000000000001e-05 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624783+0000", 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9555000000000003e-05 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624838+0000", 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4728000000000002e-05 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624927+0000", 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9493000000000005e-05 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624955+0000", 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7966999999999998e-05 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625702+0000", 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074682700000000004 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625741+0000", 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8572000000000003e-05 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.908833491999999, 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010216209999999999, 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.612 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628126+0000", 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1699999999999998e-07 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628133+0000", 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0639999999999996e-06 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628143+0000", 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628149+0000", 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628197+0000", 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8272000000000001e-05 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628217+0000", 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0171999999999999e-05 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629111+0000", 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089439299999999995 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629147+0000", 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5317000000000001e-05 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.616 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 64.796563805000005, 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001056343, 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:16.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7899999999999999e-07 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740399+0000", 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1389999999999997e-06 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740411+0000", 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1313e-05 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740415+0000", 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1269999999999996e-06 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740462+0000", 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6659999999999997e-05 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740475+0000", 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3301e-05 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741430+0000", 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095528300000000002 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741451+0000", 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1041000000000001e-05 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 64.796128651000004, 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099133399999999997, 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740831+0000", 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740929+0000", 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8220000000000005e-05 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740943+0000", 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4121000000000001e-05 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740947+0000", 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3250000000000001e-06 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740990+0000", 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2842000000000001e-05 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741010+0000", 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0168000000000001e-05 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741802+0000", 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079172499999999996 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741821+0000", 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9434999999999999e-05 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.624 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 64.795032453999994, 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098299900000000003, 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:16.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741927+0000", 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3299999999999998e-07 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741932+0000", 2024-09-11T08:44:16.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741938+0000", 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6099999999999997e-06 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741941+0000", 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.27e-06 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741971+0000", 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9708000000000001e-05 2024-09-11T08:44:16.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741982+0000", 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1148000000000001e-05 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742880+0000", 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089861100000000001 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742909+0000", 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8824999999999999e-05 2024-09-11T08:44:16.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.629 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:16.629 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:16.629 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:16.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:16.631 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-11T08:44:16.631 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:16.630+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:16.730 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:16.731+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:16.831 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:16.832+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:16.846 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:16.846 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:16.846 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:16.846 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:16.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:16.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:16.932 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:16.932+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:16.996 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:16.996 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:16.996 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:16.996 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:16.996 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:17.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:17.053 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:17.053+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:17.103 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:17.122 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:17.122 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:17.122 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:17.122 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:17.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:17.140 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:17.140 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:17.141 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:17.141+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:17.158 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:17.178 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:17.196 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:17.197 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:17.215 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:17.233 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:17.234 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:17.234 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:17.234+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:17.281 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:17.281 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:17.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:17.334 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:17.334+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:17.435 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:17.435+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:17.442 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:17.442 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:17.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:17.535 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:17.535+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:17.613 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:17.613 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:17.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:17.666 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:17.667+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:17.736 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:17.737+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:17.832 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:17.832 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:17.837 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:17.837+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:17.845 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:17.938 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:17.938+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:17.994 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:17.995 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:18.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:18.045 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:18.046+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:18.140 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:18.140+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:18.155 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:18.155 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:18.165 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:18.241 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:18.241+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:18.280 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:18.280 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:18.289 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:18.342 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:18.342+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:18.392 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:18.392 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:18.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:18.442 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:18.443+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:18.543 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:18.544+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:18.545 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:18.545 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:18.554 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:18.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T08:44:18.644 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:18.645+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:18.697 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:18.697 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:18.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:18.745 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:18.746+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:18.846 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:18.846+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:18.848 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:18.849 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:18.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:18.947 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:18.947+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:18.960 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:18.960 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:18.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:19.047 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:19.048+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:19.113 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:19.113 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:19.122 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:19.147 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:19.148+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:19.224 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:19.224 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:19.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:19.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:19.276 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:19.277+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:19.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:19.378 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:19.378 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:19.386 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:19.387+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:19.397 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:19.413 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:19.413 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:19.431 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:19.431 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:19.448 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:19.449 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:19.449 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:19.450+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:19.465 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:19.466 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:19.490 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:19.490 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:19.496 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:19.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:19.526 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:19.551 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:19.551+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:19.595 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-11T08:44:19.644 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:19.644 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:19.644 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:19.644 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:19.645 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:19.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:19.657 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:19.658+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:19.753 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:19.753+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:19.758 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:19.758 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:19.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:19.758 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:19.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.397583322000003, 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:19.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:44:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:19.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:44:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.765 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.323480959999998, 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063100159999999999, 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 35 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434236+0000", 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.05e-07 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434278+0000", 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2227000000000001e-05 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434283+0000", 2024-09-11T08:44:19.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.31e-06 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434299+0000", 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5982999999999999e-05 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436429+0000", 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297619999999999 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436479+0000", 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9789000000000001e-05 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.436582+0000", 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102923 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437975+0000", 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001393502 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440467+0000", 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024918449999999999 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440509+0000", 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1575000000000001e-05 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440546+0000", 2024-09-11T08:44:19.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6894999999999998e-05 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.323409471999994, 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063275370000000003, 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434307+0000", 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.46e-07 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434310+0000", 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8140000000000001e-06 2024-09-11T08:44:19.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434321+0000", 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0829000000000001e-05 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434325+0000", 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2570000000000001e-06 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437030+0000", 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027052159999999999 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437083+0000", 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2026e-05 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437164+0000", 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1169999999999997e-05 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438351+0000", 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001187137 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440555+0000", 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022038119999999999 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440598+0000", 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3340999999999997e-05 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440635+0000", 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6588999999999998e-05 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.323385861000006, 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064069330000000001, 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434331+0000", 2024-09-11T08:44:19.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3200000000000001e-07 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434334+0000", 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6529999999999998e-06 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434338+0000", 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3089999999999997e-06 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.434341+0000", 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8009999999999999e-06 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437546+0000", 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032056630000000001 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437594+0000", 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7401000000000002e-05 2024-09-11T08:44:19.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.437673+0000", 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9509000000000001e-05 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.438552+0000", 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087918800000000004 2024-09-11T08:44:19.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440642+0000", 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002089774 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440703+0000", 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699000000000003e-05 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.440738+0000", 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4703999999999998e-05 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.281618477999999, 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.785 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:19.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:44:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.281594612000006, 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:44:19.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:44:19.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:44:19.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.281488554000006, 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.797 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:44:19.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:44:19.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.803 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.129786077999995, 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052701520000000002, 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 113 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.804 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627930+0000", 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627931+0000", 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627935+0000", 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0709999999999996e-06 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627943+0000", 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.729e-06 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627949+0000", 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7590000000000003e-06 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631870+0000", 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039215539999999998 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631916+0000", 2024-09-11T08:44:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5878999999999999e-05 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.631985+0000", 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8688000000000004e-05 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632786+0000", 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080139299999999996 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633144+0000", 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035765299999999999 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633169+0000", 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5171999999999999e-05 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633201+0000", 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1566000000000002e-05 2024-09-11T08:44:19.809 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.129756982000004, 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066454779999999998, 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.810 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 114 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:19.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627960+0000", 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627964+0000", 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627970+0000", 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5060000000000001e-06 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627975+0000", 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9910000000000002e-06 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632819+0000", 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048437369999999999 2024-09-11T08:44:19.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632871+0000", 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1348999999999999e-05 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.632943+0000", 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2463000000000001e-05 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634095+0000", 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011517000000000001 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634557+0000", 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046178099999999998 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634582+0000", 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5327e-05 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634605+0000", 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3152000000000001e-05 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.129730643000002, 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.816 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:44:19.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.129661343999999, 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:44:19.822 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:44:19.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:44:19.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:44:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.129636391999995, 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.828 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:19.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:44:19.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.719939256000004, 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:44:19.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.719851642999998, 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.840 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:44:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:44:19.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.719812134999998, 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:44:19.846 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:44:19.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:44:19.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.71978593, 2024-09-11T08:44:19.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:44:19.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.719734674999998, 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.858 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:44:19.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.719722755999996, 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.864 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:19.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:44:19.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:44:19.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:44:19.869 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.719708878999995, 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.870 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.719667337000004, 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:19.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:44:19.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:19.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:44:19.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:44:19.882 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.882 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.882 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.882 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:19.882 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:19.882 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:19.884 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:19.884+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:19.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:19.984 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:19.984+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:20.042 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:20.043 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:20.043 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:20.043 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:20.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:20.096 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:20.097+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:20.186 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:20.187+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:20.207 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:20.207 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:20.207 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:20.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:20.287 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:20.287+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:20.329 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-11T08:44:20.329 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.904703670999993, 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011043540000000001, 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423247+0000", 2024-09-11T08:44:20.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423249+0000", 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0200000000000001e-06 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423401+0000", 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015192100000000001 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423432+0000", 2024-09-11T08:44:20.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1282999999999998e-05 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423439+0000", 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.241e-06 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423512+0000", 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3620000000000003e-05 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.423529+0000", 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6813999999999999e-05 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424295+0000", 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076553700000000005 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.424352+0000", 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6918e-05 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.893274887999993, 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011035800000000001, 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:20.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434676+0000", 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434677+0000", 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434683+0000", 2024-09-11T08:44:20.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.212e-06 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434701+0000", 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8067000000000002e-05 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434721+0000", 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9562999999999999e-05 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434778+0000", 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7581000000000003e-05 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.434800+0000", 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2204e-05 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435738+0000", 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093753900000000004 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435780+0000", 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1683000000000001e-05 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.892811528999999, 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.892291740000005, 2024-09-11T08:44:20.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010649730000000001, 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435659+0000", 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435660+0000", 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.51e-07 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435665+0000", 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435674+0000", 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:44:20.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435679+0000", 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6539999999999998e-06 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435815+0000", 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013680800000000001 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435838+0000", 2024-09-11T08:44:20.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2821000000000001e-05 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436692+0000", 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085374500000000005 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436724+0000", 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2246000000000003e-05 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.891747366000004, 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:44:20.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:44:20.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:44:20.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'15)", 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.891253496000004, 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108423, 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436697+0000", 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436698+0000", 2024-09-11T08:44:20.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4499999999999996e-07 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436737+0000", 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9226e-05 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436751+0000", 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3172e-05 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436758+0000", 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689999999999999e-06 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436816+0000", 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7506999999999997e-05 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436836+0000", 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0324000000000001e-05 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437746+0000", 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000910537 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437782+0000", 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5150000000000001e-05 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:38 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-11T08:44:20.355 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.886761449000005, 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010117800000000001, 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441189+0000", 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441190+0000", 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9999999999999996e-07 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441257+0000", 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6607999999999997e-05 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441269+0000", 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1953999999999999e-05 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441275+0000", 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9020000000000004e-06 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441315+0000", 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0506999999999997e-05 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441328+0000", 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3165e-05 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442139+0000", 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081040300000000001 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442201+0000", 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2440999999999994e-05 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:39 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'25, mlcod=16'22)", 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.886327234999996, 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010054160000000001, 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441624+0000", 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0900000000000002e-07 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441721+0000", 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6898000000000006e-05 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441734+0000", 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.332e-05 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441744+0000", 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.465e-06 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441812+0000", 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7694999999999995e-05 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.441834+0000", 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2614000000000001e-05 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442593+0000", 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075926599999999998 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.442629+0000", 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5648999999999997e-05 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:53 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'31, mlcod=16'29)", 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.874951690000003, 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098036799999999991, 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.364 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.452999+0000", 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453000+0000", 2024-09-11T08:44:20.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0400000000000005e-07 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453012+0000", 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1911e-05 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453024+0000", 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2257e-05 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453030+0000", 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0009999999999998e-06 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453076+0000", 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5516000000000003e-05 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453096+0000", 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0361e-05 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453934+0000", 2024-09-11T08:44:20.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083800900000000002 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453980+0000", 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5509000000000001e-05 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.874379817000005, 2024-09-11T08:44:20.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001121478, 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453571+0000", 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453572+0000", 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1600000000000001e-07 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453577+0000", 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4199999999999998e-06 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453588+0000", 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1134999999999999e-05 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453594+0000", 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453654+0000", 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9202e-05 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.453668+0000", 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4375999999999999e-05 2024-09-11T08:44:20.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454647+0000", 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097895199999999995 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454693+0000", 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5558e-05 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.873826926000007, 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001004079, 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454124+0000", 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454125+0000", 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2399999999999997e-07 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454212+0000", 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6881999999999999e-05 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454234+0000", 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2109000000000001e-05 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454251+0000", 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7552999999999999e-05 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454294+0000", 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2910000000000001e-05 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.454306+0000", 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1853e-05 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455087+0000", 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078128799999999999 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.455128+0000", 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.066e-05 2024-09-11T08:44:20.376 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.871889549000002, 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010907619999999999, 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456061+0000", 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456062+0000", 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7899999999999998e-07 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456134+0000", 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1617000000000002e-05 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456145+0000", 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1581e-05 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456156+0000", 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0671e-05 2024-09-11T08:44:20.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456200+0000", 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4159999999999997e-05 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.456217+0000", 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7061000000000001e-05 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457093+0000", 2024-09-11T08:44:20.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087593800000000004 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.457152+0000", 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9055000000000003e-05 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.864023799999998, 2024-09-11T08:44:20.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:44:20.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:44:20.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.863601207000002, 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001143493, 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:20.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2799999999999996e-07 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464395+0000", 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4858000000000003e-05 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464409+0000", 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3553000000000001e-05 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464416+0000", 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2899999999999997e-06 2024-09-11T08:44:20.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464487+0000", 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0994000000000005e-05 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464520+0000", 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2790000000000003e-05 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465459+0000", 2024-09-11T08:44:20.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093904100000000005 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465493+0000", 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4338999999999998e-05 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:20.390 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.850953067999995, 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001017771, 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.476998+0000", 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7700000000000004e-07 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477043+0000", 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4397999999999997e-05 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477053+0000", 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0565e-05 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477063+0000", 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6169999999999996e-06 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477138+0000", 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5370000000000005e-05 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477151+0000", 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2962999999999999e-05 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.477978+0000", 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082701499999999996 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.478016+0000", 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7166000000000003e-05 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:99 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'65, mlcod=16'63)", 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.703242497000005, 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010323559999999999, 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624708+0000", 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624715+0000", 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9739999999999996e-06 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624744+0000", 2024-09-11T08:44:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8240000000000001e-05 2024-09-11T08:44:20.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624783+0000", 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9555000000000003e-05 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624838+0000", 2024-09-11T08:44:20.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4728000000000002e-05 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624927+0000", 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9493000000000005e-05 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.624955+0000", 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7966999999999998e-05 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625702+0000", 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074682700000000004 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.625741+0000", 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8572000000000003e-05 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.402 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.699825708000006, 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010216209999999999, 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628125+0000", 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628126+0000", 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1699999999999998e-07 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628133+0000", 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0639999999999996e-06 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628143+0000", 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628149+0000", 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628197+0000", 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8272000000000001e-05 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.628217+0000", 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0171999999999999e-05 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629111+0000", 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089439299999999995 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.629147+0000", 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5317000000000001e-05 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:154 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'108, mlcod=16'105)", 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 68.587556020999997, 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001056343, 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740395+0000", 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7899999999999999e-07 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740399+0000", 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1389999999999997e-06 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740411+0000", 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1313e-05 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740415+0000", 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1269999999999996e-06 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740462+0000", 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6659999999999997e-05 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740475+0000", 2024-09-11T08:44:20.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3301e-05 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741430+0000", 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095528300000000002 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741451+0000", 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1041000000000001e-05 2024-09-11T08:44:20.412 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:155 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'109, mlcod=16'105)", 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 68.587120866999996, 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099133399999999997, 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740830+0000", 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740831+0000", 2024-09-11T08:44:20.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740929+0000", 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8220000000000005e-05 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740943+0000", 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4121000000000001e-05 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740947+0000", 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3250000000000001e-06 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.740990+0000", 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2842000000000001e-05 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741010+0000", 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0168000000000001e-05 2024-09-11T08:44:20.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741802+0000", 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079172499999999996 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741821+0000", 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9434999999999999e-05 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:157 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 68.58602467, 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098299900000000003, 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:20.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741926+0000", 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741927+0000", 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3299999999999998e-07 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741932+0000", 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741938+0000", 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6099999999999997e-06 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741941+0000", 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.27e-06 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741971+0000", 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9708000000000001e-05 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.741982+0000", 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1148000000000001e-05 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742880+0000", 2024-09-11T08:44:20.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089861100000000001 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:11.742909+0000", 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8824999999999999e-05 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.422 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:20.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:20.425 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:20.424+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:20.524 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:20.525+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:20.544 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:20.544 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:20.544 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:20.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:20.638 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:20.639+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:20.701 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:44:20.701 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:44:20.701 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-11T08:44:20.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-11T08:44:20.705 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:20.706 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:20.706 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:20.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:20.759 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:20.760+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:20.828 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:20.828+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:20.877 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:20.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 62.488625806000002, 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095305400000000005, 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:20.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7299999999999996e-07 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389734+0000", 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-05 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389772+0000", 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7768999999999997e-05 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389783+0000", 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839000000000001e-05 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389852+0000", 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9268000000000002e-05 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389880+0000", 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7597999999999999e-05 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390619+0000", 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073945300000000005 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390671+0000", 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 61.484216539999998, 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095448600000000005, 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:20.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9399999999999999e-07 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394137+0000", 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0331e-05 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394155+0000", 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7558999999999999e-05 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394170+0000", 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4591e-05 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394237+0000", 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7026999999999994e-05 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394257+0000", 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0145000000000001e-05 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395041+0000", 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078400400000000004 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395081+0000", 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0635e-05 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(2.2 35/34 [PushOp(2:5cea7035:::rbd_info:head, version: 21'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@21'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 34.386113186999999, 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044003299999999999, 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:20.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:20.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492231+0000", 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9400000000000005e-07 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492245+0000", 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4115e-05 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492265+0000", 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9585e-05 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:20.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492282+0000", 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7322000000000001e-05 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492387+0000", 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010562599999999999 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492396+0000", 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4889999999999995e-06 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:20.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492670+0000", 2024-09-11T08:44:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027430199999999998 2024-09-11T08:44:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:20.891 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:20.891 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:20.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:20.935 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:20.936+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:21.003 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:21.003 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:21.003 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:21.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:21.053 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:21.054+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:21.130 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:21.131+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:21.171 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:21.171 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:21.171 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:21.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:21.228 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-11T08:44:21.229+0000 7f1a550d0640 -1 mon.a@0(leader).osd e53 definitely_dead 0 2024-09-11T08:44:21.231 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:21.231+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:21.332 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:21.332+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:21.334 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-11T08:44:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.951120176000003, 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025475459999999999, 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383750+0000", 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383766+0000", 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5752000000000001e-05 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383780+0000", 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4122e-05 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383881+0000", 2024-09-11T08:44:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010105 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383914+0000", 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2999e-05 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386246+0000", 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023319579999999999 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386291+0000", 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4435999999999999e-05 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(1.0 44/43 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.790462569999999, 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001134822, 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544644+0000", 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00024222499999999999 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544669+0000", 2024-09-11T08:44:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4910999999999999e-05 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544703+0000", 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4557000000000001e-05 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544795+0000", 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1619999999999994e-05 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544805+0000", 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0287e-05 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.545535+0000", 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073039999999999997 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(1.0 44/43 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.787538046000002, 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010332119999999999, 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547326+0000", 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6700000000000003e-07 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547537+0000", 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021147200000000001 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547561+0000", 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3702000000000001e-05 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547579+0000", 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8348000000000001e-05 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547690+0000", 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110136 2024-09-11T08:44:21.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547704+0000", 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.471e-05 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.548358+0000", 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065407699999999998 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4228.0:2 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.162550710000001, 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00026821700000000001, 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4228", 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 2 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:21.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172314+0000", 2024-09-11T08:44:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7680000000000001e-06 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172322+0000", 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1070000000000005e-06 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172336+0000", 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3402e-05 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172348+0000", 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2666999999999999e-05 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172444+0000", 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5519999999999993e-05 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172581+0000", 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013675300000000001 2024-09-11T08:44:21.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:21.352 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:21.352 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:21.353 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:21.434 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:21.434+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:21.504 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:21.504 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:21.504 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:21.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:21.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:21.558 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:21.559+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:21.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T08:44:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:marked out osd.6. 2024-09-11T08:44:21.635 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:21.635+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:21.643 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:21.644 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:21.644 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:21.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:21.743 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:21.743 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:21.743 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:21.743+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:21.809 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:21.837 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:21.837+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:21.843 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:21.844 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:21.844 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:21.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:21.877 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:21.909 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:21.938 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:21.939+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:21.944 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:21.970 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:21.970 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:21.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:21.980 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:22.039 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:22.040+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:22.124 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:22.124 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:22.132 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:22.140 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:22.141+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:22.236 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:22.244 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:22.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:22.246 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:22.246+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:22.346 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:22.347+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:22.390 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:22.390 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:22.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:22.447 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:22.448+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:22.543 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:22.543 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:22.548 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:22.548+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:22.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:22.629 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-11T08:44:22.648 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:22.649+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:22.697 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:22.698 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:22.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:22.749 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:22.749+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:22.850 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:22.850 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:22.850 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:22.850+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:22.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:22.950 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:22.951+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:23.002 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:23.003 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:23.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:23.053 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:23.053+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:23.153 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:23.154 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:23.154 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:23.154+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:23.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:23.255 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:23.255+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:23.265 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:23.265 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:23.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:23.358 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:23.359+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:23.418 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:23.419 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:23.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:23.469 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:23.469+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:23.558 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:23.558+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:23.570 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:23.570 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:23.579 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:23.658 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:23.659+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:23.725 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:23.726 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:23.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:23.758 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:23.759+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:23.859 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:23.860+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:23.887 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:23.887 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:23.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:23.960 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:23.961+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:23.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:24.008 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:24.008 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:24.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:24.062 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:24.063+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:24.157 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:24.162 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:24.162+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:24.192 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:24.192 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:24.207 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:24.207 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:24.207 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:24.207 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:24.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:24.224 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:24.256 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:24.257 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:24.263 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:24.264+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:24.287 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:24.287 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:24.319 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:24.349 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:24.364 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:24.364+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:24.380 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:24.380 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:24.393 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:24.393 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:24.393 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:24.393 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:24.393 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:24.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:24.489 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:24.490+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:24.557 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 66.168732163000001, 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095305400000000005, 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389718+0000", 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7299999999999996e-07 2024-09-11T08:44:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389734+0000", 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-05 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389772+0000", 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7768999999999997e-05 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389783+0000", 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839000000000001e-05 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389852+0000", 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9268000000000002e-05 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.389880+0000", 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7597999999999999e-05 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390619+0000", 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073945300000000005 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:18.390671+0000", 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.164322897000005, 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00095448600000000005, 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394127+0000", 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9399999999999999e-07 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394137+0000", 2024-09-11T08:44:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0331e-05 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394155+0000", 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7558999999999999e-05 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394170+0000", 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4591e-05 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394237+0000", 2024-09-11T08:44:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7026999999999994e-05 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.394257+0000", 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0145000000000001e-05 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395041+0000", 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078400400000000004 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:19.395081+0000", 2024-09-11T08:44:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0635e-05 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(2.2 35/34 [PushOp(2:5cea7035:::rbd_info:head, version: 21'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@21'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 38.066219543999999, 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044003299999999999, 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492230+0000", 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492231+0000", 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9400000000000005e-07 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492245+0000", 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4115e-05 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492265+0000", 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9585e-05 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492282+0000", 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7322000000000001e-05 2024-09-11T08:44:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492387+0000", 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010562599999999999 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492396+0000", 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4889999999999995e-06 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:46.492670+0000", 2024-09-11T08:44:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027430199999999998 2024-09-11T08:44:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:24.571 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:24.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:24.573 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:24.573+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:24.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T08:44:24.699 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:24.700+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:24.726 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:24.726 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:24.726 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:24.726 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:24.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:24.777 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:24.778+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:24.874 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:24.875+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:24.887 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:24.887 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:24.896 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:24.975 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:24.976+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4269.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 67.624199078000004, 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025475459999999999, 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383743+0000", 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383750+0000", 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383766+0000", 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5752000000000001e-05 2024-09-11T08:44:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383780+0000", 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4122e-05 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383881+0000", 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010105 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.383914+0000", 2024-09-11T08:44:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2999e-05 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386246+0000", 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023319579999999999 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:17.386291+0000", 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4435999999999999e-05 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(1.0 44/43 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.463541471999999, 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001134822, 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544401+0000", 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544644+0000", 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00024222499999999999 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544669+0000", 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4910999999999999e-05 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544703+0000", 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4557000000000001e-05 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544795+0000", 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1619999999999994e-05 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.544805+0000", 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0287e-05 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.545535+0000", 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073039999999999997 2024-09-11T08:44:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(1.0 44/43 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.460616947999998, 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010332119999999999, 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547325+0000", 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547326+0000", 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6700000000000003e-07 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547537+0000", 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021147200000000001 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547561+0000", 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3702000000000001e-05 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547579+0000", 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8348000000000001e-05 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547690+0000", 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110136 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.547704+0000", 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.471e-05 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:01.548358+0000", 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065407699999999998 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4228.0:2 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.835629611999998, 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00026821700000000001, 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4228", 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 2 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172312+0000", 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172314+0000", 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7680000000000001e-06 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172322+0000", 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1070000000000005e-06 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172336+0000", 2024-09-11T08:44:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3402e-05 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172348+0000", 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2666999999999999e-05 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172444+0000", 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5519999999999993e-05 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:02.172581+0000", 2024-09-11T08:44:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013675300000000001 2024-09-11T08:44:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:25.024 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:25.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:25.075 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:25.076+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:25.135 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:25.136 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:25.136 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:25.136 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:25.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:25.188 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:25.189+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:25.253 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:25.253 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:25.254 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:25.254 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:25.254 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:25.262 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:25.304 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:25.304+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:25.378 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:25.379+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:25.411 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-11T08:44:25.411 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:25.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:25.479 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:25.479+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:25.536 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:25.536 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:25.536 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:25.536 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:25.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:25.586 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:25.587+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:25.634 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-11T08:44:25.680 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:25.680+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:25.690 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:25.691 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:25.699 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:25.781 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:25.781+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:25.801 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:25.801 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:25.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:25.801 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:44:25.801 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:25.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:25.881 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:25.882+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:25.914 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:25.914 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:25.914 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:25.914 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:25.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:26.007 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:26.008+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:26.066 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:26.067 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:26.067 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:26.067 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:26.067 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:26.075 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:26.083 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:26.083+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:26.183 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:26.184+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.840165115999994, 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:44:26.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:26.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.227 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.834979688999994, 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1896000000000003e-05, 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386388+0000", 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386389+0000", 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-09-11T08:44:26.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386394+0000", 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1839999999999998e-06 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386405+0000", 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386412+0000", 2024-09-11T08:44:26.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9530000000000002e-06 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386453+0000", 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.074e-05 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386460+0000", 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4479999999999997e-06 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386470+0000", 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.829889707000007, 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000246512, 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.232 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 2 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391478+0000", 2024-09-11T08:44:26.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391480+0000", 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1129999999999999e-06 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391492+0000", 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1749999999999999e-05 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391522+0000", 2024-09-11T08:44:26.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9886000000000002e-05 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391541+0000", 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9453000000000002e-05 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391624+0000", 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3307999999999996e-05 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391724+0000", 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100002 2024-09-11T08:44:26.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.829396639999999, 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018776700000000001, 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391971+0000", 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391972+0000", 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391985+0000", 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3173000000000001e-05 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.391995+0000", 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392003+0000", 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2449999999999998e-06 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392046+0000", 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2630999999999999e-05 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.392159+0000", 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113069 2024-09-11T08:44:26.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 67.833735376000007, 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:26.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:44:26.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:26.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:26.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:26.247 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 67.830597522000005, 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018276100000000001, 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390770+0000", 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390771+0000", 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4099999999999997e-07 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390778+0000", 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2230000000000001e-06 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390789+0000", 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.081e-05 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390795+0000", 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9070000000000004e-06 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390869+0000", 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4010000000000005e-05 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390876+0000", 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390953+0000", 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6971000000000006e-05 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.251 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.829486191000001, 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:26.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4269.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.826198528000006, 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104585, 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:26.258 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395169+0000", 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395170+0000", 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1009999999999999e-06 2024-09-11T08:44:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395177+0000", 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395188+0000", 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0849999999999999e-05 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395194+0000", 2024-09-11T08:44:26.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3999999999999997e-06 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395254+0000", 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0359000000000003e-05 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395261+0000", 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3679999999999998e-06 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395274+0000", 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.29e-05 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(2.2 35/34 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.728030670000003, 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000180452, 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493337+0000", 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3599999999999999e-07 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493342+0000", 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493350+0000", 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8990000000000001e-06 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493359+0000", 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8759999999999994e-06 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493397+0000", 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8303999999999998e-05 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:46.493517+0000", 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012013199999999999 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4228.0:1 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.050583416999999, 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001851562, 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4228", 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:26.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170786+0000", 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9180000000000001e-06 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:26.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170796+0000", 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6649999999999998e-06 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170808+0000", 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1970999999999999e-05 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170818+0000", 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0682e-05 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170899+0000", 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0235999999999995e-05 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170904+0000", 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.921e-06 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172541+0000", 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001637021 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:26.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172636+0000", 2024-09-11T08:44:26.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5148000000000004e-05 2024-09-11T08:44:26.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.270 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:26.270 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:26.270 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:26.271 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:26.313 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:26.313+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:26.385 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:26.386+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:26.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:26.414 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:26.415 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:26.415 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:26.415 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:26.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:26.485 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:26.486+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:26.519 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:26.520 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:26.527 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:26.527 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:26.527 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:26.527 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:26.527 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:26.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:26.537 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:26.537 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:26.554 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:26.554 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:26.571 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:26.571 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:26.587 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:26.605 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:26.605 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:26.620 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:26.621+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:26.622 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:26.642 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:26.643 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 7, 1] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:44:26.643 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:44:26.644 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-11T08:44:26.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-11T08:44:26.681 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:26.681 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:26.686 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:26.687+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:26.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:26.787 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:26.788+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:26.834 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:26.834 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:26.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:26.888 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:26.888+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:26.947 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-11T08:44:26.947+0000 7f1a550d0640 -1 mon.a@0(leader).osd e56 definitely_dead 0 2024-09-11T08:44:26.988 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:26.988 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:26.988 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:26.989+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:26.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:27.047 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 94% (12257/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-11T08:44:27.089 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:27.090+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:27.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:27.191 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:27.191+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:27.219 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:27.219 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:27.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:27.291 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:27.292+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:27.337 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:27.337 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:27.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:27.393 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:27.393+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to 'daaf78de334647fc0c800f4ac495cc9475a26bae'. 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-09-11T08:44:27.446 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-11T08:44:27.447 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-09-11T08:44:27.447 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-11T08:44:27.447 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-09-11T08:44:27.447 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-11T08:44:27.447 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T08:44:27.447 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-11T08:44:27.447 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at daaf78de334 Merge pull request #59682 from zdover23/wip-doc-2024-09-10-backport-59677-to-squid 2024-09-11T08:44:27.451 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T08:44:27.455 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:27.455 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:27.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:27.479 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T08:44:27.481 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T08:44:27.481 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T08:44:27.494 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:27.494+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:27.586 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:27.586 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:27.595 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:27.595+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:27.596 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:marked in osd.6. 2024-09-11T08:44:27.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T08:44:27.654 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-11T08:44:27.695 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:27.696+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:27.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:27.798 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:27.799+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:27.864 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:27.864 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:27.873 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:27.897 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:27.897+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:27.997 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:27.998+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:28.022 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:28.022 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:28.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:28.099 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:28.099+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:28.139 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:28.139 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:28.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:28.200 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:28.200+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:28.229 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T08:44:28.260 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:28.260 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:28.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:28.305 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T08:44:28.306 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:28.306+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:28.346 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T08:44:28.348 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T08:44:28.348 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T08:44:28.379 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:28.379 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:28.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:28.389 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T08:44:28.392 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-11T08:44:28.392 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T08:44:28.402 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:28.402+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:28.449 INFO:tasks.workunit:Running workunits matching rbd/journal.sh on client.0... 2024-09-11T08:44:28.450 INFO:tasks.workunit:Running workunit rbd/journal.sh... 2024-09-11T08:44:28.450 DEBUG:teuthology.orchestra.run.smithi045:workunit test rbd/journal.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=daaf78de334647fc0c800f4ac495cc9475a26bae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/journal.sh 2024-09-11T08:44:28.512 INFO:tasks.workunit.client.0.smithi045.stderr:stty: 'standard input': Inappropriate ioctl for device 2024-09-11T08:44:28.512 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:28.512+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:28.518 INFO:tasks.workunit.client.0.smithi045.stderr:+ test_rbd_journal 2024-09-11T08:44:28.518 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdjournal47013 2024-09-11T08:44:28.518 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd create --image-feature exclusive-lock --image-feature journaling --size 128 testrbdjournal47013 2024-09-11T08:44:28.540 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:28.540 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:28.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:28.632 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:28.633+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:28.646 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-11T08:44:28.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:28.700 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:28.700 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:28.704 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:28.705+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:28.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:28.780 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:28.798 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:28.798 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:28.805 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:28.806+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:28.814 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:28.823 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:28.823 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:28.830 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:28.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:28.847 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:28.863 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:28.864 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:28.880 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:28.897 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:28.906 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:28.906+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:28.951 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd info testrbdjournal47013 --format=xml 2024-09-11T08:44:28.951 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //image/journal 2024-09-11T08:44:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:28.977 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:28.978 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:28.978 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:28.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:29.008 INFO:tasks.workunit.client.0.smithi045.stderr:+ local journal=1332e735c7c0 2024-09-11T08:44:29.008 INFO:tasks.workunit.client.0.smithi045.stderr:+ test -n 1332e735c7c0 2024-09-11T08:44:29.008 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal info 1332e735c7c0 2024-09-11T08:44:29.030 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:29.031+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:29.041 INFO:tasks.workunit.client.0.smithi045.stdout:rbd journal '1332e735c7c0': 2024-09-11T08:44:29.041 INFO:tasks.workunit.client.0.smithi045.stdout: header_oid: journal.1332e735c7c0 2024-09-11T08:44:29.041 INFO:tasks.workunit.client.0.smithi045.stdout: object_oid_prefix: journal_data.2.1332e735c7c0. 2024-09-11T08:44:29.041 INFO:tasks.workunit.client.0.smithi045.stdout: order: 24 (16 MiB objects) 2024-09-11T08:44:29.041 INFO:tasks.workunit.client.0.smithi045.stdout: splay_width: 4 2024-09-11T08:44:29.044 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal info --journal 1332e735c7c0 2024-09-11T08:44:29.077 INFO:tasks.workunit.client.0.smithi045.stdout:rbd journal '1332e735c7c0': 2024-09-11T08:44:29.078 INFO:tasks.workunit.client.0.smithi045.stdout: header_oid: journal.1332e735c7c0 2024-09-11T08:44:29.078 INFO:tasks.workunit.client.0.smithi045.stdout: object_oid_prefix: journal_data.2.1332e735c7c0. 2024-09-11T08:44:29.078 INFO:tasks.workunit.client.0.smithi045.stdout: order: 24 (16 MiB objects) 2024-09-11T08:44:29.078 INFO:tasks.workunit.client.0.smithi045.stdout: splay_width: 4 2024-09-11T08:44:29.080 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal info --image testrbdjournal47013 2024-09-11T08:44:29.091 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [] 2024-09-11T08:44:29.091 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:29.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:29.108 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:29.109+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:29.128 INFO:tasks.workunit.client.0.smithi045.stdout:rbd journal '1332e735c7c0': 2024-09-11T08:44:29.128 INFO:tasks.workunit.client.0.smithi045.stdout: header_oid: journal.1332e735c7c0 2024-09-11T08:44:29.128 INFO:tasks.workunit.client.0.smithi045.stdout: object_oid_prefix: journal_data.2.1332e735c7c0. 2024-09-11T08:44:29.128 INFO:tasks.workunit.client.0.smithi045.stdout: order: 24 (16 MiB objects) 2024-09-11T08:44:29.128 INFO:tasks.workunit.client.0.smithi045.stdout: splay_width: 4 2024-09-11T08:44:29.131 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd feature disable testrbdjournal47013 journaling 2024-09-11T08:44:29.189 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd info testrbdjournal47013 --format=xml 2024-09-11T08:44:29.189 INFO:tasks.workunit.client.0.smithi045.stderr:+ expect_false xmlstarlet sel -t -v //image/journal 2024-09-11T08:44:29.189 INFO:tasks.workunit.client.0.smithi045.stderr:+ set -x 2024-09-11T08:44:29.189 INFO:tasks.workunit.client.0.smithi045.stderr:+ xmlstarlet sel -t -v //image/journal 2024-09-11T08:44:29.208 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:29.209+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:29.233 INFO:tasks.workunit.client.0.smithi045.stderr:+ return 0 2024-09-11T08:44:29.233 INFO:tasks.workunit.client.0.smithi045.stderr:+ expect_false rbd journal info 1332e735c7c0 2024-09-11T08:44:29.233 INFO:tasks.workunit.client.0.smithi045.stderr:+ set -x 2024-09-11T08:44:29.233 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal info 1332e735c7c0 2024-09-11T08:44:29.246 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:29.246 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:29.246 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:29.246 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:29.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:29.266 INFO:tasks.workunit.client.0.smithi045.stderr:failed to get journal metadata: (2) No such file or directory 2024-09-11T08:44:29.266 INFO:tasks.workunit.client.0.smithi045.stderr:rbd: journal info: (2) No such file or directory 2024-09-11T08:44:29.269 INFO:tasks.workunit.client.0.smithi045.stderr:+ return 0 2024-09-11T08:44:29.269 INFO:tasks.workunit.client.0.smithi045.stderr:+ expect_false rbd journal info --image testrbdjournal47013 2024-09-11T08:44:29.269 INFO:tasks.workunit.client.0.smithi045.stderr:+ set -x 2024-09-11T08:44:29.269 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal info --image testrbdjournal47013 2024-09-11T08:44:29.309 INFO:tasks.workunit.client.0.smithi045.stderr:rbd: journaling is not enabled for image testrbdjournal47013 2024-09-11T08:44:29.309 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:29.310+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:29.312 INFO:tasks.workunit.client.0.smithi045.stderr:+ return 0 2024-09-11T08:44:29.312 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd feature enable testrbdjournal47013 journaling 2024-09-11T08:44:29.359 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:29.360 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:29.360 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:29.360 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:29.360 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:29.367 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd info testrbdjournal47013 --format=xml 2024-09-11T08:44:29.367 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //image/journal 2024-09-11T08:44:29.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:29.410 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:29.411+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:29.412 INFO:tasks.workunit.client.0.smithi045.stderr:+ local journal1=1332e735c7c0 2024-09-11T08:44:29.412 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 1332e735c7c0 = 1332e735c7c0 2024-09-11T08:44:29.412 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal info 1332e735c7c0 2024-09-11T08:44:29.445 INFO:tasks.workunit.client.0.smithi045.stdout:rbd journal '1332e735c7c0': 2024-09-11T08:44:29.445 INFO:tasks.workunit.client.0.smithi045.stdout: header_oid: journal.1332e735c7c0 2024-09-11T08:44:29.445 INFO:tasks.workunit.client.0.smithi045.stdout: object_oid_prefix: journal_data.2.1332e735c7c0. 2024-09-11T08:44:29.445 INFO:tasks.workunit.client.0.smithi045.stdout: order: 24 (16 MiB objects) 2024-09-11T08:44:29.445 INFO:tasks.workunit.client.0.smithi045.stdout: splay_width: 4 2024-09-11T08:44:29.448 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal status 1332e735c7c0 2024-09-11T08:44:29.481 INFO:tasks.workunit.client.0.smithi045.stdout:minimum_set: 0 2024-09-11T08:44:29.481 INFO:tasks.workunit.client.0.smithi045.stdout:active_set: 0 2024-09-11T08:44:29.481 INFO:tasks.workunit.client.0.smithi045.stdout:registered clients: 2024-09-11T08:44:29.481 INFO:tasks.workunit.client.0.smithi045.stdout: [id=, commit_position=[positions=[]], state=connected] 2024-09-11T08:44:29.484 INFO:tasks.workunit.client.0.smithi045.stderr:+ local count=10 2024-09-11T08:44:29.484 INFO:tasks.workunit.client.0.smithi045.stderr:+ save_commit_position 1332e735c7c0 2024-09-11T08:44:29.484 INFO:tasks.workunit.client.0.smithi045.stderr:+ local journal=1332e735c7c0 2024-09-11T08:44:29.484 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p rbd getomapval journal.1332e735c7c0 client_ /tmp/rbd_journal47013/1332e735c7c0.client_.omap 2024-09-11T08:44:29.511 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:29.511+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 72.133331119999994, 2024-09-11T08:44:29.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074093429999999997, 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381203+0000", 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381207+0000", 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9929999999999997e-06 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381213+0000", 2024-09-11T08:44:29.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381227+0000", 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.364e-05 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381239+0000", 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909999999999999e-05 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381297+0000", 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8162999999999999e-05 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.381307+0000", 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7040000000000006e-06 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383365+0000", 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020584879999999998 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383487+0000", 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121724 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.383582+0000", 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5168999999999996e-05 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.386464+0000", 2024-09-11T08:44:29.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028820009999999999 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388510+0000", 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020459060000000001 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388560+0000", 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9993000000000001e-05 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:17.388612+0000", 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2062999999999999e-05 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 71.126901380000007, 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003311881, 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.522 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 5 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387632+0000", 2024-09-11T08:44:29.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387634+0000", 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1490000000000001e-06 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387646+0000", 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1751e-05 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387665+0000", 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8643000000000001e-05 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387674+0000", 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2799999999999992e-06 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387733+0000", 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8968999999999997e-05 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.387738+0000", 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389279+0000", 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001541202 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389371+0000", 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1325999999999999e-05 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.389474+0000", 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103459 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390325+0000", 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085096399999999995 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390879+0000", 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055436099999999996 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390913+0000", 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3834999999999997e-05 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:18.390944+0000", 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1059000000000002e-05 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4269.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-11T08:44:29.528 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.122652195000001, 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034912509999999999, 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4269", 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/160353431", 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391881+0000", 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391883+0000", 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8670000000000001e-06 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391892+0000", 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391908+0000", 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.643e-05 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391918+0000", 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1600000000000004e-06 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391984+0000", 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6179000000000005e-05 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.391989+0000", 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-06 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393740+0000", 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001750796 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393836+0000", 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6817000000000005e-05 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.393955+0000", 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011846 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395266+0000", 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001310802 2024-09-11T08:44:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395308+0000", 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1999999999999998e-05 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395344+0000", 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6287e-05 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:19.395373+0000", 2024-09-11T08:44:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8822999999999998e-05 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4228.0:1 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.343749420999998, 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001851562, 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4228", 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1160345166", 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170784+0000", 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170786+0000", 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9180000000000001e-06 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170796+0000", 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6649999999999998e-06 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170808+0000", 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1970999999999999e-05 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170818+0000", 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0682e-05 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170899+0000", 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0235999999999995e-05 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.170904+0000", 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.921e-06 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172541+0000", 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001637021 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:02.172636+0000", 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5148000000000004e-05 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4914.0:7 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:28.942921+0000", 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.57161262700000004, 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001488198, 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4914", 2024-09-11T08:44:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1101648607", 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942921+0000", 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942921+0000", 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942922+0000", 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8700000000000004e-07 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942928+0000", 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.75e-06 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942939+0000", 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0910999999999999e-05 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942944+0000", 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3090000000000003e-06 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942989+0000", 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5167999999999999e-05 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.943084+0000", 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4817999999999999e-05 2024-09-11T08:44:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.943234+0000", 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014992599999999999 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.943931+0000", 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00069681099999999998 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.944313+0000", 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00038260399999999998 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.944370+0000", 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7111999999999998e-05 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.944409+0000", 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8902000000000001e-05 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4745.0:3 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:28.988281+0000", 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.52625253100000002, 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00044139199999999998, 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4745", 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1736183952", 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-09-11T08:44:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988281+0000", 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988281+0000", 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988283+0000", 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5349999999999999e-06 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988290+0000", 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9539999999999998e-06 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988298+0000", 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7819999999999999e-06 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988315+0000", 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6759e-05 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988378+0000", 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3281000000000006e-05 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988471+0000", 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2854000000000002e-05 2024-09-11T08:44:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.988723+0000", 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00025122700000000003 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4745.0:7 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:28.989988+0000", 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.52454591100000003, 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046921700000000002, 2024-09-11T08:44:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4745", 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1736183952", 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.989988+0000", 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.989988+0000", 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.989989+0000", 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.37e-07 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.989993+0000", 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1200000000000004e-06 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.990000+0000", 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6869999999999994e-06 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.990005+0000", 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9989999999999999e-06 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.990045+0000", 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9776e-05 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.990125+0000", 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9443000000000005e-05 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.990457+0000", 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000332455 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4766.0:12 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [watch watch cookie 140049756817808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.173766+0000", 2024-09-11T08:44:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.34076799000000002, 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010676870000000001, 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4766", 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1468394574", 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 12 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173766+0000", 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173766+0000", 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173766+0000", 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2200000000000004e-07 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173769+0000", 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3140000000000002e-06 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173775+0000", 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.314e-06 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173778+0000", 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4089999999999999e-06 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173808+0000", 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9153999999999999e-05 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173856+0000", 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8905e-05 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.173946+0000", 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9736999999999998e-05 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.174543+0000", 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00059645 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.174766+0000", 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022296699999999999 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.174794+0000", 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8467999999999999e-05 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.174833+0000", 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9347000000000001e-05 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4766.0:14 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.175240+0000", 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.33929400599999998, 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00087741299999999996, 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4766", 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1468394574", 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175240+0000", 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175240+0000", 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175240+0000", 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3300000000000002e-07 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175243+0000", 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7410000000000001e-06 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175249+0000", 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2020000000000001e-06 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175253+0000", 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3610000000000001e-06 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175282+0000", 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.938e-05 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175321+0000", 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9165999999999997e-05 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175419+0000", 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7961000000000006e-05 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.175971+0000", 2024-09-11T08:44:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055171599999999995 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.176067+0000", 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5871000000000004e-05 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.176094+0000", 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7596000000000001e-05 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.176117+0000", 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2785999999999998e-05 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4766.0:31 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call rbd.set_features in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.183132+0000", 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.33140198999999998, 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00095927800000000004, 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4766", 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1468394574", 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 31 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183132+0000", 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183132+0000", 2024-09-11T08:44:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183133+0000", 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1500000000000003e-07 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183138+0000", 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0799999999999996e-06 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183150+0000", 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1989e-05 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183162+0000", 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2734999999999999e-05 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183203+0000", 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0836e-05 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183254+0000", 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0704000000000001e-05 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183365+0000", 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111543 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.183801+0000", 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00043540799999999998 2024-09-11T08:44:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184040+0000", 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023904199999999999 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184067+0000", 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7334e-05 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184091+0000", 2024-09-11T08:44:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3791999999999999e-05 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4766.0:34 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.184699+0000", 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.32983487099999997, 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00089307600000000003, 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4766", 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1468394574", 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 34 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184699+0000", 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184699+0000", 2024-09-11T08:44:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184699+0000", 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6300000000000005e-07 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184702+0000", 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6230000000000001e-06 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184708+0000", 2024-09-11T08:44:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.105e-06 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184711+0000", 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.303e-06 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184741+0000", 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9516e-05 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184782+0000", 2024-09-11T08:44:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1032999999999998e-05 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.184882+0000", 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9885999999999999e-05 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185332+0000", 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00044971799999999999 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185542+0000", 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000210337 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185568+0000", 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5942999999999999e-05 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185592+0000", 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4049e-05 2024-09-11T08:44:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4766.0:36 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [watch unwatch cookie 140049756817808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.185713+0000", 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.328821048, 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011042459999999999, 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4766", 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1468394574", 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 36 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185713+0000", 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185713+0000", 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185713+0000", 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0600000000000001e-07 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185714+0000", 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.296e-06 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185719+0000", 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5090000000000004e-06 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185722+0000", 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1420000000000002e-06 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185869+0000", 2024-09-11T08:44:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014738500000000001 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185947+0000", 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7899999999999996e-05 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186095+0000", 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014780300000000001 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186569+0000", 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047413199999999999 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186732+0000", 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000163108 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186758+0000", 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6279e-05 2024-09-11T08:44:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186817+0000", 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8386e-05 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4766.0:37 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.185912+0000", 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.32862127600000002, 2024-09-11T08:44:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001116751, 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4766", 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1468394574", 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185912+0000", 2024-09-11T08:44:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185912+0000", 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185913+0000", 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9599999999999999e-07 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185921+0000", 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4069999999999993e-06 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185927+0000", 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3929999999999999e-06 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.185931+0000", 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0879999999999997e-06 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186358+0000", 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00042732399999999998 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for rw locks", 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186399+0000", 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0694e-05 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186869+0000", 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047056399999999998 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.186944+0000", 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4318000000000002e-05 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.187029+0000", 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5366999999999998e-05 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4781.0:12 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [watch watch cookie 139806487103712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.354107+0000", 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.16042698799999999, 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001119735, 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4781", 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1227408613", 2024-09-11T08:44:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 12 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354107+0000", 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354107+0000", 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354107+0000", 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1399999999999997e-07 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354110+0000", 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5780000000000001e-06 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354117+0000", 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2200000000000003e-06 2024-09-11T08:44:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354121+0000", 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2719999999999999e-06 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354152+0000", 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0558000000000001e-05 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354195+0000", 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2861999999999997e-05 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354287+0000", 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2534000000000005e-05 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.354845+0000", 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055813200000000003 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355146+0000", 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030076300000000001 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355176+0000", 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9533e-05 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355226+0000", 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0769000000000001e-05 2024-09-11T08:44:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4781.0:14 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.355607+0000", 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.15892694299999999, 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00097089099999999998, 2024-09-11T08:44:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4781", 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1227408613", 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355607+0000", 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355607+0000", 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355607+0000", 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.06e-07 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355610+0000", 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4219999999999999e-06 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355616+0000", 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5490000000000003e-06 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355620+0000", 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5379999999999998e-06 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355648+0000", 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8062999999999999e-05 2024-09-11T08:44:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355688+0000", 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9737000000000003e-05 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.355782+0000", 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4778000000000006e-05 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.356172+0000", 2024-09-11T08:44:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00038981399999999999 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.356528+0000", 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035597100000000001 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.356557+0000", 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.902e-05 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.356578+0000", 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0492999999999998e-05 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4781.0:25 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call rbd.set_features in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.361294+0000", 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.15323966999999999, 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092247100000000001, 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4781", 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1227408613", 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361294+0000", 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361294+0000", 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361295+0000", 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.61e-07 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361300+0000", 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4240000000000001e-06 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361311+0000", 2024-09-11T08:44:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0879e-05 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361319+0000", 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4880000000000001e-06 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361359+0000", 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0556999999999998e-05 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361406+0000", 2024-09-11T08:44:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6470000000000001e-05 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361509+0000", 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010341899999999999 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.361895+0000", 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00038595399999999998 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362146+0000", 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00025116100000000002 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362178+0000", 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2302999999999999e-05 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362216+0000", 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8055000000000001e-05 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4781.0:28 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.362810+0000", 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.15172375900000001, 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085770699999999998, 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4781", 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1227408613", 2024-09-11T08:44:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 28 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362810+0000", 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362810+0000", 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362811+0000", 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2900000000000003e-07 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362813+0000", 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7049999999999999e-06 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362820+0000", 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2600000000000002e-06 2024-09-11T08:44:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362823+0000", 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4800000000000001e-06 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362858+0000", 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5203999999999997e-05 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362898+0000", 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0040999999999998e-05 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.362992+0000", 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3449000000000006e-05 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363368+0000", 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037666899999999999 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363601+0000", 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023289 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363632+0000", 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0318999999999999e-05 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363668+0000", 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6060999999999999e-05 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4781.0:30 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [watch unwatch cookie 139806487103712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.363799+0000", 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.150735165, 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001318893, 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4781", 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1227408613", 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 30 2024-09-11T08:44:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363799+0000", 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363799+0000", 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363799+0000", 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0499999999999999e-07 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363801+0000", 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3209999999999999e-06 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363808+0000", 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7009999999999999e-06 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363814+0000", 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2670000000000003e-06 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.363955+0000", 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000140727 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364005+0000", 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0331999999999999e-05 2024-09-11T08:44:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364100+0000", 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4994999999999994e-05 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364526+0000", 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00042589499999999997 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.365038+0000", 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00051143800000000002 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.365065+0000", 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7671999999999999e-05 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.365117+0000", 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2139999999999999e-05 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4781.0:31 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.364000+0000", 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.150534103, 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013001880000000001, 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4781", 2024-09-11T08:44:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1227408613", 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 31 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364000+0000", 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364000+0000", 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364000+0000", 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.61e-07 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364004+0000", 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0119999999999999e-06 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364012+0000", 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7270000000000007e-06 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364019+0000", 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0310000000000001e-06 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364278+0000", 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00025876300000000002 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for rw locks", 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.364344+0000", 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6256999999999999e-05 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.365158+0000", 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00081423599999999997 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.365226+0000", 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7720999999999998e-05 2024-09-11T08:44:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.365300+0000", 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3679999999999999e-05 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4787.0:7 2.3 2:c6246a82:::rbd_header.1332e735c7c0:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.406809+0000", 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.107724746, 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000472962, 2024-09-11T08:44:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4787", 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1587186956", 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.406809+0000", 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.406809+0000", 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.406810+0000", 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6000000000000003e-07 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.406814+0000", 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1509999999999997e-06 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.406825+0000", 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0677e-05 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.406830+0000", 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0950000000000003e-06 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.406872+0000", 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1949000000000001e-05 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.406951+0000", 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9418000000000004e-05 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.407282+0000", 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033091199999999999 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:29.629 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:29.630 INFO:tasks.workunit.client.0.smithi045.stderr:Writing to /tmp/rbd_journal47013/1332e735c7c0.client_.omap 2024-09-11T08:44:29.630 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd bench --io-type write testrbdjournal47013 --io-size 4096 --io-threads 1 --io-total 40960 --io-pattern seq 2024-09-11T08:44:29.631 INFO:tasks.workunit.client.0.smithi045.stdout:bench type write io_size 4096 io_threads 1 bytes 40960 pattern sequential 2024-09-11T08:44:29.631 INFO:tasks.workunit.client.0.smithi045.stdout: SEC OPS OPS/SEC BYTES/SEC 2024-09-11T08:44:29.631 INFO:tasks.workunit.client.0.smithi045.stdout:elapsed: 0 ops: 10 ops/sec: 833.347 bytes/sec: 3.3 MiB/s 2024-09-11T08:44:29.632 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal status --image testrbdjournal47013 2024-09-11T08:44:29.632 INFO:tasks.workunit.client.0.smithi045.stderr:+ fgrep tid=9 2024-09-11T08:44:29.632 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:29.632 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:29.631+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:29.652 INFO:tasks.workunit.client.0.smithi045.stdout: [id=, commit_position=[positions=[[object_number=3, tag_tid=1, entry_tid=11], [object_number=2, tag_tid=1, entry_tid=10], [object_number=1, tag_tid=1, entry_tid=9], [object_number=0, tag_tid=1, entry_tid=8]]], state=connected] 2024-09-11T08:44:29.652 INFO:tasks.workunit.client.0.smithi045.stderr:+ restore_commit_position 1332e735c7c0 2024-09-11T08:44:29.652 INFO:tasks.workunit.client.0.smithi045.stderr:+ local journal=1332e735c7c0 2024-09-11T08:44:29.653 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p rbd setomapval journal.1332e735c7c0 client_ 2024-09-11T08:44:29.690 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal status --image testrbdjournal47013 2024-09-11T08:44:29.691 INFO:tasks.workunit.client.0.smithi045.stderr:+ fgrep 'positions=[]' 2024-09-11T08:44:29.731 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:29.731+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:29.741 INFO:tasks.workunit.client.0.smithi045.stdout: [id=, commit_position=[positions=[]], state=connected] 2024-09-11T08:44:29.742 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd journal inspect --verbose 1332e735c7c0 2024-09-11T08:44:29.742 INFO:tasks.workunit.client.0.smithi045.stderr:++ grep -c 'event_type.*AioWrite' 2024-09-11T08:44:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:29.778 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:29.778 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:29.778 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:29.784 INFO:tasks.workunit.client.0.smithi045.stderr:2024-09-11T08:44:29.785+0000 7f6857fff640 -1 JournalMetadata: 0x560905dd48f0 failed to locate client: INSPECT 2024-09-11T08:44:29.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:29.787 INFO:tasks.workunit.client.0.smithi045.stderr:+ local count1=10 2024-09-11T08:44:29.788 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 10 -eq 10 2024-09-11T08:44:29.788 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal export 1332e735c7c0 /tmp/rbd_journal47013/journal.export 2024-09-11T08:44:29.827 INFO:tasks.workunit.client.0.smithi045.stdout:12 entries processed, 0 errors 2024-09-11T08:44:29.830 INFO:tasks.workunit.client.0.smithi045.stderr:2024-09-11T08:44:29.830+0000 7fe5c3fff640 -1 JournalMetadata: 0x5580382682a0 failed to locate client: EXPORT 2024-09-11T08:44:29.832 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:29.833+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:29.833 INFO:tasks.workunit.client.0.smithi045.stderr:++ stat -c %s /tmp/rbd_journal47013/journal.export 2024-09-11T08:44:29.834 INFO:tasks.workunit.client.0.smithi045.stderr:+ local size=55926 2024-09-11T08:44:29.834 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 55926 -gt 0 2024-09-11T08:44:29.834 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd export testrbdjournal47013 /tmp/rbd_journal47013/testrbdjournal47013.export 2024-09-11T08:44:29.892 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:29.892 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:29.893 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:29.893 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:29.893 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:29.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:29.904 INFO:tasks.workunit.client.0.smithi045.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-09-11T08:44:29.909 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image1=testrbdjournal470131 2024-09-11T08:44:29.909 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd create --image-feature exclusive-lock --image-feature journaling --size 128 testrbdjournal470131 2024-09-11T08:44:29.933 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:29.934+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:29.964 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd info testrbdjournal470131 --format=xml 2024-09-11T08:44:29.964 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //image/journal 2024-09-11T08:44:30.009 INFO:tasks.workunit.client.0.smithi045.stderr:+ journal1=13daa594a2f0 2024-09-11T08:44:30.009 INFO:tasks.workunit.client.0.smithi045.stderr:+ save_commit_position 13daa594a2f0 2024-09-11T08:44:30.009 INFO:tasks.workunit.client.0.smithi045.stderr:+ local journal=13daa594a2f0 2024-09-11T08:44:30.009 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p rbd getomapval journal.13daa594a2f0 client_ /tmp/rbd_journal47013/13daa594a2f0.client_.omap 2024-09-11T08:44:30.034 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:30.035+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:30.045 INFO:tasks.workunit.client.0.smithi045.stderr:Writing to /tmp/rbd_journal47013/13daa594a2f0.client_.omap 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4914.0:6 2.6 e58/57 2:678275a5:::rbd_id.testrbdjournal47013:head v 58'1, mlcod=0'0)", 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:28.941127+0000", 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 1.1061194240000001, 2024-09-11T08:44:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015120800000000001, 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941127+0000", 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941127+0000", 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941128+0000", 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2099999999999995e-07 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941134+0000", 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9349999999999999e-06 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941145+0000", 2024-09-11T08:44:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0679999999999999e-05 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941152+0000", 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941201+0000", 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8789999999999999e-05 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941224+0000", 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3781000000000002e-05 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942607+0000", 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013825790000000001 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942639+0000", 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2495e-05 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5076.0:19 2.7 2:f5ec7188:::rbd_data.1332e735c7c0.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.877380+0000", 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.16986675300000001, 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000212015, 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5076", 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/732424200", 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 19 2024-09-11T08:44:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.877380+0000", 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.877380+0000", 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.877382+0000", 2024-09-11T08:44:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5269999999999999e-06 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.877387+0000", 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1989999999999997e-06 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.877398+0000", 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1552000000000001e-05 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.877408+0000", 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8449999999999998e-06 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.877479+0000", 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0795000000000002e-05 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.877592+0000", 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113097 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5076.0:30 2.7 2:e8bb06a3:::rbd_data.1332e735c7c0.0000000000000012:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.890957+0000", 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.15628943000000001, 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000178431, 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5076", 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/732424200", 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 30 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.890957+0000", 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.890957+0000", 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.890958+0000", 2024-09-11T08:44:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7300000000000004e-07 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.890964+0000", 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.631e-06 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.890975+0000", 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1056999999999999e-05 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.890982+0000", 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3649999999999998e-06 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.891044+0000", 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1388000000000002e-05 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.891136+0000", 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2016999999999997e-05 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5076.0:33 2.7 2:e5e11d0e:::rbd_data.1332e735c7c0.0000000000000015:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.894333+0000", 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.152913718, 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017110699999999999, 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5076", 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/732424200", 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 33 2024-09-11T08:44:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.894333+0000", 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.894333+0000", 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.894334+0000", 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9699999999999995e-07 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.894338+0000", 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1160000000000001e-06 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.894349+0000", 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0759e-05 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.894357+0000", 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7989999999999994e-06 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.894411+0000", 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4048000000000002e-05 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.894504+0000", 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3587999999999999e-05 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.5082.0:7 2.6 e58/57 2:64b93aac:::rbd_header.13daa594a2f0:head v 58'2, mlcod=0'0)", 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.956290+0000", 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.090956698000000002, 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00067193199999999998, 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956290+0000", 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956290+0000", 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956291+0000", 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0300000000000001e-06 2024-09-11T08:44:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956304+0000", 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2712e-05 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956317+0000", 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3557000000000001e-05 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956325+0000", 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2389999999999996e-06 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956381+0000", 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6419999999999999e-05 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956404+0000", 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2889999999999999e-05 2024-09-11T08:44:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956926+0000", 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00052176499999999999 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.956962+0000", 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6319000000000002e-05 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.068 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:30.068 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal import --dest testrbdjournal470131 /tmp/rbd_journal47013/journal.export 2024-09-11T08:44:30.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:30.101 INFO:tasks.workunit.client.0.smithi045.stdout:12 entries processed, 0 errors 2024-09-11T08:44:30.101 INFO:tasks.workunit.client.0.smithi045.stdout:Waiting for journal append to complete... 2024-09-11T08:44:30.109 INFO:tasks.workunit.client.0.smithi045.stderr:2024-09-11T08:44:30.110+0000 7f4efcc1a640 -1 JournalMetadata: 0x562c3b9be680 failed to locate client: IMPORT 2024-09-11T08:44:30.112 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd snap create testrbdjournal470131@test 2024-09-11T08:44:30.135 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:30.136+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:30.177 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:30.177 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:30.186 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:30.237 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:30.237+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:30.289 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:30.290 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:30.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:30.337 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:30.338+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:30.438 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:30.439+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 81.082867609000004, 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:44:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.966902765, 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:44:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:44:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.966878898999994, 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:44:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:44:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.966772840999994, 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:44:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.815014930000004, 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:44:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:44:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.814945631000001, 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:44:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.814920678999997, 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:44:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:44:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.405223543000005, 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:44:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.40513593, 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:44:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:44:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.405096422, 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:44:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:44:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:44:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.405070217000002, 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:44:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:44:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.405018962, 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:44:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:44:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.405007042999998, 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:44:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.404993165999997, 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:44:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.404951624000006, 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:44:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:44:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:51 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 20480~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586049+0000", 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.856951566, 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075292390000000001, 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 51 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586049+0000", 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586049+0000", 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586050+0000", 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5999999999999999e-07 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586053+0000", 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2059999999999999e-06 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586057+0000", 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7900000000000001e-06 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586059+0000", 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7429999999999998e-06 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.587728+0000", 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001668262 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.587809+0000", 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1908000000000003e-05 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.587944+0000", 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013496100000000001 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589343+0000", 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013987299999999999 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593521+0000", 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004178106 2024-09-11T08:44:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593555+0000", 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3312999999999998e-05 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593578+0000", 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3859999999999999e-05 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:52 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 24576~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586121+0000", 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.856879999, 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075068119999999999, 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 52 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586121+0000", 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586121+0000", 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586121+0000", 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7899999999999999e-07 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586125+0000", 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7639999999999999e-06 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586130+0000", 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6750000000000001e-06 2024-09-11T08:44:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586139+0000", 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0820000000000005e-06 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588358+0000", 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022190539999999998 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588434+0000", 2024-09-11T08:44:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6458000000000006e-05 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588561+0000", 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012707699999999999 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589387+0000", 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00082586699999999999 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593585+0000", 2024-09-11T08:44:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041978629999999996 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593609+0000", 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3903000000000001e-05 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593628+0000", 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8689999999999999e-05 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:53 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 28672~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586168+0000", 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.85683257199999996, 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075023590000000001, 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 53 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586168+0000", 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586168+0000", 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586169+0000", 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5100000000000001e-07 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586173+0000", 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1489999999999996e-06 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586177+0000", 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4530000000000004e-06 2024-09-11T08:44:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586180+0000", 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9210000000000001e-06 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588909+0000", 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002728744 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588957+0000", 2024-09-11T08:44:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7728000000000001e-05 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589034+0000", 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7041999999999999e-05 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.590298+0000", 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012642829999999999 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593631+0000", 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003333005 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593654+0000", 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2668e-05 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593671+0000", 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7014999999999999e-05 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:54 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 32768~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586185+0000", 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.856815628, 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075261379999999999, 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586185+0000", 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586185+0000", 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586185+0000", 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4499999999999998e-07 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586189+0000", 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4520000000000002e-06 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586192+0000", 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2310000000000001e-06 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586196+0000", 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4790000000000001e-06 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589421+0000", 2024-09-11T08:44:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032254060000000001 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589499+0000", 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7651000000000006e-05 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589633+0000", 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013448099999999999 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.590472+0000", 2024-09-11T08:44:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00083839899999999998 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593673+0000", 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00320193 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593695+0000", 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1557e-05 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593711+0000", 2024-09-11T08:44:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6307000000000002e-05 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:55 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 36864~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586199+0000", 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.85680175000000003, 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075524069999999997, 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 55 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586199+0000", 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586199+0000", 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586199+0000", 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586203+0000", 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9790000000000004e-06 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586207+0000", 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3919999999999999e-06 2024-09-11T08:44:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586209+0000", 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5469999999999999e-06 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589949+0000", 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037401380000000001 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589993+0000", 2024-09-11T08:44:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3665000000000003e-05 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.590065+0000", 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1649000000000005e-05 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.590784+0000", 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00071953399999999997 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593715+0000", 2024-09-11T08:44:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029311250000000001 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593736+0000", 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0669000000000001e-05 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593751+0000", 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5488000000000001e-05 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:30.566 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:30.567 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:30.567+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:30.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:30.658 INFO:tasks.workunit.client.0.smithi045.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-11T08:44:30.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T08:44:30.666 INFO:tasks.workunit.client.0.smithi045.stderr:+ restore_commit_position 13daa594a2f0 2024-09-11T08:44:30.666 INFO:tasks.workunit.client.0.smithi045.stderr:+ local journal=13daa594a2f0 2024-09-11T08:44:30.666 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p rbd setomapval journal.13daa594a2f0 client_ 2024-09-11T08:44:30.668 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:30.669+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:30.682 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:30.682 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:30.682 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:30.683 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:30.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:30.706 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal inspect --image testrbdjournal470131 --verbose 2024-09-11T08:44:30.706 INFO:tasks.workunit.client.0.smithi045.stderr:+ awk ' 2024-09-11T08:44:30.706 INFO:tasks.workunit.client.0.smithi045.stderr: /AioFlush/ {a++} # match: "event_type": "AioFlush", 2024-09-11T08:44:30.706 INFO:tasks.workunit.client.0.smithi045.stderr: /AioWrite/ {w++} # match: "event_type": "AioWrite", 2024-09-11T08:44:30.706 INFO:tasks.workunit.client.0.smithi045.stderr: /SnapCreate/ {s++} # match: "event_type": "SnapCreate", 2024-09-11T08:44:30.706 INFO:tasks.workunit.client.0.smithi045.stderr: /OpFinish/ {f++} # match: "event_type": "OpFinish", 2024-09-11T08:44:30.707 INFO:tasks.workunit.client.0.smithi045.stderr: /entries inspected/ {t=$1; e=$4} # match: 14 entries inspected, 0 errors 2024-09-11T08:44:30.707 INFO:tasks.workunit.client.0.smithi045.stderr: {print} # for diagnostic 2024-09-11T08:44:30.707 INFO:tasks.workunit.client.0.smithi045.stderr: END { 2024-09-11T08:44:30.707 INFO:tasks.workunit.client.0.smithi045.stderr: if (a != 2 || w != 10 || s != 1 || f != 1 || t != 14 || e != 0) exit(1) 2024-09-11T08:44:30.707 INFO:tasks.workunit.client.0.smithi045.stderr: } 2024-09-11T08:44:30.707 INFO:tasks.workunit.client.0.smithi045.stderr: ' 2024-09-11T08:44:30.769 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:30.769+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:30.772 INFO:tasks.workunit.client.0.smithi045.stderr:2024-09-11T08:44:30.773+0000 7f2cee6b2640 -1 JournalMetadata: 0x55e321551d00 failed to locate client: INSPECT 2024-09-11T08:44:30.776 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=1 2024-09-11T08:44:30.776 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.776 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioFlush", 2024-09-11T08:44:30.776 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.580928+0000" 2024-09-11T08:44:30.776 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.776 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=2 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 0, 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582298+0000" 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=3 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 4096, 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582344+0000" 2024-09-11T08:44:30.777 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=4 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 8192, 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582393+0000" 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=5 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 12288, 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.778 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582440+0000" 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=6 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 16384, 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582497+0000" 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=7 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 20480, 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.779 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582531+0000" 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=8 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 24576, 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582567+0000" 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=9 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 28672, 2024-09-11T08:44:30.780 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582612+0000" 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=10 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 32768, 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582646+0000" 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=11 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioWrite", 2024-09-11T08:44:30.781 INFO:tasks.workunit.client.0.smithi045.stdout: "offset": 36864, 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout: "length": 4096, 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582681+0000" 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=12 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "AioFlush", 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:29.582718+0000" 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=13 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "SnapCreate", 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout: "op_tid": 1, 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout: "snap_name": "test", 2024-09-11T08:44:30.782 INFO:tasks.workunit.client.0.smithi045.stdout: "snapshot_namespace_type": "user", 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:30.171993+0000" 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout:Entry: tag_id=1, commit_tid=14 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout:{ 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout: "event_type": "OpFinish", 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout: "op_tid": 1, 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout: "op_tid": 1, 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout: "result": 0, 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout: "timestamp": "2024-09-11T08:44:30.654996+0000" 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout:} 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout:Summary: 2024-09-11T08:44:30.783 INFO:tasks.workunit.client.0.smithi045.stdout: 14 entries inspected, 0 errors 2024-09-11T08:44:30.784 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd export testrbdjournal470131@test /tmp/rbd_journal47013/testrbdjournal470131.export 2024-09-11T08:44:30.825 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:30.826 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:30.826 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:30.826 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:30.826 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:30.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:30.849 INFO:tasks.workunit.client.0.smithi045.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-09-11T08:44:30.855 INFO:tasks.workunit.client.0.smithi045.stderr:+ cmp /tmp/rbd_journal47013/testrbdjournal47013.export /tmp/rbd_journal47013/testrbdjournal470131.export 2024-09-11T08:44:30.870 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:30.871+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:30.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:30.954 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal reset 1332e735c7c0 2024-09-11T08:44:30.957 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:30.957 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:30.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:30.957 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:30.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.957 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:44:30.957 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.520432638000003, 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:44:30.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.519368474999993, 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:44:30.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:44:30.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.491644909000001, 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:44:30.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:44:30.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.491222316000005, 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001143493, 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464350+0000", 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2799999999999996e-07 2024-09-11T08:44:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464395+0000", 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4858000000000003e-05 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464409+0000", 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3553000000000001e-05 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464416+0000", 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2899999999999997e-06 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464487+0000", 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0994000000000005e-05 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464520+0000", 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2790000000000003e-05 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465459+0000", 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093904100000000005 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465493+0000", 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4338999999999998e-05 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:38 2.5 e58/18 2:acb3f4d3:::journal_data.2.1332e735c7c0.1:head v 58'3, mlcod=58'1)", 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.582839+0000", 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.3727327069999999, 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001582017, 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582839+0000", 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582839+0000", 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582840+0000", 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8599999999999998e-07 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582851+0000", 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1270999999999999e-05 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582860+0000", 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1379999999999996e-06 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582866+0000", 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0109999999999997e-06 2024-09-11T08:44:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582926+0000", 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9422999999999997e-05 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582946+0000", 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0404e-05 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584382+0000", 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00143597 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584421+0000", 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9113999999999999e-05 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:41 2.4 e58/35 2:2460a7a9:::journal_data.2.1332e735c7c0.0:head v 58'18, mlcod=58'15)", 2024-09-11T08:44:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.583188+0000", 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.372384429, 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001191037, 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583188+0000", 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583188+0000", 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583188+0000", 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9500000000000002e-07 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583195+0000", 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9249999999999998e-06 2024-09-11T08:44:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583213+0000", 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8130000000000001e-05 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583218+0000", 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.091e-06 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583280+0000", 2024-09-11T08:44:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1999000000000001e-05 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583301+0000", 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0849999999999999e-05 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584339+0000", 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010380949999999999 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584379+0000", 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9351999999999999e-05 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:40 2.5 e58/18 2:b55ab9c6:::journal_data.2.1332e735c7c0.3:head v 58'4, mlcod=58'1)", 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.583367+0000", 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.3722048849999999, 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001571233, 2024-09-11T08:44:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583367+0000", 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583367+0000", 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583368+0000", 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9100000000000004e-07 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583389+0000", 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1639000000000001e-05 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583396+0000", 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0369999999999997e-06 2024-09-11T08:44:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583400+0000", 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9160000000000003e-06 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583449+0000", 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9159999999999997e-05 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583472+0000", 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2376000000000001e-05 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584862+0000", 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001390358 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584938+0000", 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6155999999999994e-05 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:42 2.5 e58/18 2:acb3f4d3:::journal_data.2.1332e735c7c0.1:head v 58'5, mlcod=58'1)", 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.583728+0000", 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.371844286, 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00137009, 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583728+0000", 2024-09-11T08:44:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583728+0000", 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583728+0000", 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6100000000000001e-07 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583782+0000", 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4169999999999998e-05 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583798+0000", 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5838e-05 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583803+0000", 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.763e-06 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583855+0000", 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2157000000000002e-05 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583873+0000", 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8170999999999999e-05 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585066+0000", 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00119308 2024-09-11T08:44:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585098+0000", 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1350000000000003e-05 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:43 2.4 e58/35 2:2460a7a9:::journal_data.2.1332e735c7c0.0:head v 58'19, mlcod=58'15)", 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.583887+0000", 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.371684795, 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012037809999999999, 2024-09-11T08:44:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583887+0000", 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583887+0000", 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583888+0000", 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9699999999999995e-07 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583902+0000", 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3665999999999999e-05 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583908+0000", 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7229999999999998e-06 2024-09-11T08:44:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583914+0000", 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2619999999999998e-06 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583976+0000", 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1972999999999998e-05 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583991+0000", 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5574e-05 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585053+0000", 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00106178 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585091+0000", 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8105999999999997e-05 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:45 2.5 e58/18 2:b55ab9c6:::journal_data.2.1332e735c7c0.3:head v 58'6, mlcod=58'1)", 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.584254+0000", 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.371317777, 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012160579999999999, 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584254+0000", 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584254+0000", 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584255+0000", 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2500000000000005e-07 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584278+0000", 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3468999999999998e-05 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584292+0000", 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.359e-05 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584300+0000", 2024-09-11T08:44:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3189999999999995e-06 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584451+0000", 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000151094 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584471+0000", 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9196000000000001e-05 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585435+0000", 2024-09-11T08:44:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00096490900000000001 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585470+0000", 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4856e-05 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5115.0:10 2.1 2:9f7ad37c:::journal_data.2.13daa594a2f0.3:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.101343+0000", 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.85422932900000004, 2024-09-11T08:44:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021259709999999999, 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5115", 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2941479348", 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 10 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101343+0000", 2024-09-11T08:44:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101343+0000", 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101344+0000", 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7999999999999993e-07 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101353+0000", 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.884e-06 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101366+0000", 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3040999999999999e-05 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101371+0000", 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.468e-06 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101414+0000", 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2929999999999997e-05 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101526+0000", 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112456 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101674+0000", 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014795800000000001 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102713+0000", 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010386760000000001 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103347+0000", 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063360999999999995 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103423+0000", 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6784000000000002e-05 2024-09-11T08:44:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103469+0000", 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5184e-05 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5115.0:14 2.1 2:9f7ad37c:::journal_data.2.13daa594a2f0.3:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.101776+0000", 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.85379635099999995, 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002548827, 2024-09-11T08:44:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5115", 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2941479348", 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101776+0000", 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101776+0000", 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101777+0000", 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0100000000000003e-07 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101798+0000", 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1659999999999999e-05 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101811+0000", 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2510999999999999e-05 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101819+0000", 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999999e-06 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102091+0000", 2024-09-11T08:44:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027193200000000001 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102173+0000", 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2126999999999995e-05 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102356+0000", 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018356399999999999 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103503+0000", 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001146509 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104260+0000", 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075766199999999998 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104289+0000", 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8639000000000001e-05 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104324+0000", 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5352000000000003e-05 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5115.0:9 2.5 e58/18 2:a93e661c:::journal_data.2.13daa594a2f0.2:head v 58'18, mlcod=58'16)", 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.102487+0000", 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.85308503999999996, 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001490618, 2024-09-11T08:44:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102487+0000", 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102487+0000", 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102488+0000", 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6499999999999998e-07 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102497+0000", 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3929999999999997e-06 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102511+0000", 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3808999999999999e-05 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102518+0000", 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4189999999999996e-06 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102592+0000", 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4028000000000003e-05 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102611+0000", 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8383e-05 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103934+0000", 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013234539999999999 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103978+0000", 2024-09-11T08:44:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3266999999999998e-05 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5115.0:18 2.1 2:9f7ad37c:::journal_data.2.13daa594a2f0.3:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.102799+0000", 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.85277253900000005, 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001838073, 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5115", 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2941479348", 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 18 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102799+0000", 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102799+0000", 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102800+0000", 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0419999999999999e-06 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102806+0000", 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8889999999999999e-06 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102818+0000", 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1263e-05 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102829+0000", 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1119e-05 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102877+0000", 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8047000000000003e-05 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102934+0000", 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7142000000000003e-05 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103027+0000", 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3517999999999995e-05 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103962+0000", 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093438899999999999 2024-09-11T08:44:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104576+0000", 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061375999999999998 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104614+0000", 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8485999999999997e-05 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104638+0000", 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3417999999999998e-05 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5115.0:13 2.5 e58/18 2:a93e661c:::journal_data.2.13daa594a2f0.2:head v 58'19, mlcod=58'16)", 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.103151+0000", 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.85242144399999997, 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001198993, 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103151+0000", 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103151+0000", 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103151+0000", 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5499999999999998e-07 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103163+0000", 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1697e-05 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103173+0000", 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0249e-05 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103179+0000", 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1630000000000001e-06 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103230+0000", 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1149000000000001e-05 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103251+0000", 2024-09-11T08:44:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0449e-05 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104307+0000", 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010559650000000001 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104350+0000", 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2666000000000001e-05 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:57 2.1 2:9eb3d535:::journal.13daa594a2f0:head [call journal.client_commit in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.656153+0000", 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.29941857700000002, 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011277710000000001, 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 57 2024-09-11T08:44:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656153+0000", 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656153+0000", 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656156+0000", 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.131e-06 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656165+0000", 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9889999999999998e-06 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656184+0000", 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9066999999999999e-05 2024-09-11T08:44:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656194+0000", 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0219e-05 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656258+0000", 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4511999999999995e-05 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656315+0000", 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6935999999999998e-05 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656452+0000", 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013661899999999999 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.656975+0000", 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00052304300000000005 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.657192+0000", 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000217137 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.657246+0000", 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4190000000000001e-05 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.657281+0000", 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4928000000000002e-05 2024-09-11T08:44:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5133.0:1 2.1 2:9eb3d535:::journal.13daa594a2f0:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.703522+0000", 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.252050261, 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011981349999999999, 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5133", 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/1567141192", 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703522+0000", 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703522+0000", 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703523+0000", 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.719e-06 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703529+0000", 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6239999999999999e-06 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703539+0000", 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6630000000000005e-06 2024-09-11T08:44:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703547+0000", 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.704e-06 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703610+0000", 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2161000000000003e-05 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703674+0000", 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4380000000000004e-05 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.703771+0000", 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6990000000000005e-05 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.704424+0000", 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065261699999999998 2024-09-11T08:44:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.704615+0000", 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00019107099999999999 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.704678+0000", 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2948999999999996e-05 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.704720+0000", 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2256999999999999e-05 2024-09-11T08:44:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5142.0:1 2.1 2:9eb3d535:::journal.13daa594a2f0:head [call journal.client_register in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.764983+0000", 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.19058851099999999, 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012467159999999999, 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5142", 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3942304305", 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.764983+0000", 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.764983+0000", 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.764985+0000", 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.626e-06 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.764991+0000", 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6930000000000004e-06 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765000+0000", 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6099999999999995e-06 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765009+0000", 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1070000000000005e-06 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765075+0000", 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6674999999999994e-05 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765141+0000", 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5778999999999995e-05 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765328+0000", 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000186906 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765901+0000", 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000572983 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.766173+0000", 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027190800000000001 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.766202+0000", 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9564999999999999e-05 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.766230+0000", 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7864e-05 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5142.0:13 2.1 2:9eb3d535:::journal.13daa594a2f0:head [call journal.client_unregister in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-11T08:44:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.770592+0000", 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.18497954, 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013752809999999999, 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5142", 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3942304305", 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 13 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770592+0000", 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770592+0000", 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770593+0000", 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2500000000000005e-07 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770599+0000", 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7649999999999999e-06 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770610+0000", 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1447999999999999e-05 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770615+0000", 2024-09-11T08:44:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9529999999999998e-06 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770674+0000", 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8573000000000003e-05 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770761+0000", 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7198000000000001e-05 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771034+0000", 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027312000000000001 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771567+0000", 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00053294499999999995 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771916+0000", 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034918599999999997 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771946+0000", 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9335999999999998e-05 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771968+0000", 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2031999999999998e-05 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5142.0:18 2.1 2:9eb3d535:::journal.13daa594a2f0:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.772679+0000", 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.18289272000000001, 2024-09-11T08:44:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001178913, 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5142", 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3942304305", 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 18 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.772679+0000", 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.772679+0000", 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.772680+0000", 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5899999999999995e-07 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.772684+0000", 2024-09-11T08:44:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1760000000000003e-06 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.772693+0000", 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2180000000000005e-06 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.772700+0000", 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.28e-06 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.772992+0000", 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00029270400000000002 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for rw locks", 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.773069+0000", 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6088999999999996e-05 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.773627+0000", 2024-09-11T08:44:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00055896400000000001 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.773701+0000", 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3425999999999998e-05 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.773858+0000", 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015729700000000001 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:31.065 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:31.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:31.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd journal inspect --verbose 1332e735c7c0 2024-09-11T08:44:31.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ expect_false grep event_type 2024-09-11T08:44:31.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ set -x 2024-09-11T08:44:31.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ grep event_type 2024-09-11T08:44:31.067 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:31.068 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:31.068 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:31.068 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:31.069 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:31.066+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:31.073 INFO:tasks.workunit.client.0.smithi045.stderr:2024-09-11T08:44:31.073+0000 7f05b095e640 -1 JournalMetadata: 0x55d38f2ef640 failed to locate client: INSPECT 2024-09-11T08:44:31.075 INFO:tasks.workunit.client.0.smithi045.stderr:+ return 0 2024-09-11T08:44:31.076 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd snap purge testrbdjournal470131 2024-09-11T08:44:31.083 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:31.084 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:31.100 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:31.101 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:31.119 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:31.140 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:31.159 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:31.167 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:31.168+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:31.175 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:31.190 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:31.190 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:31.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:31.268 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:31.269+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:31.346 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:31.346 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:31.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:31.398 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:31.398+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:31.462 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:31.462 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:31.469 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:31.470+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:31.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:31.571 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:31.570+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:31.619 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:31.620 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:31.628 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-11T08:44:31.663 INFO:tasks.workunit.client.0.smithi045.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-09-11T08:44:31.671 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdjournal470131 2024-09-11T08:44:31.671 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:31.671+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:31.756 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete...2024-09-11T08:44:31.755+0000 7f3dede94640 0 -- 172.21.15.45:0/1386556248 >> [v2:172.21.15.86:6816/2975777507,v1:172.21.15.86:6817/2975777507] conn(0x7f3dc80012a0 msgr2=0x7f3dc8001680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-11T08:44:31.762 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete...2024-09-11T08:44:31.762+0000 7f3dee695640 0 -- 172.21.15.45:0/1386556248 >> [v2:172.21.15.45:6808/2540486849,v1:172.21.15.45:6809/2540486849] conn(0x7f3dc8011440 msgr2=0x7f3dc80138d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:31.772 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:31.772 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:31.773 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:31.772+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:31.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:31.783 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 96% complete... Removing image: 100% complete...done. 2024-09-11T08:44:31.786 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdjournal47013 2024-09-11T08:44:31.863 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete...2024-09-11T08:44:31.862+0000 7fcbffc82640 0 -- 172.21.15.45:0/1341229340 >> [v2:172.21.15.45:6808/2540486849,v1:172.21.15.45:6809/2540486849] conn(0x7fcbdc00c5b0 msgr2=0x7fcbdc00ca30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:31.867 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete...2024-09-11T08:44:31.867+0000 7fcbfea20640 0 -- 172.21.15.45:0/1341229340 >> [v2:172.21.15.86:6824/498386186,v1:172.21.15.86:6825/498386186] conn(0x56286e7bbc60 msgr2=0x56286e7be0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:31.874 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:31.873+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:31.885 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-09-11T08:44:31.888 INFO:tasks.workunit.client.0.smithi045.stderr:+ set +x 2024-09-11T08:44:31.888 INFO:tasks.workunit.client.0.smithi045.stderr:+ test_rbd_create 2024-09-11T08:44:31.888 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdcreate47013 2024-09-11T08:44:31.889 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd create --image-feature exclusive-lock --image-feature journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 --size 256 testrbdcreate47013 2024-09-11T08:44:31.904 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:31.904 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:31.914 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:31.951 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdcreate47013 'journal info' //journal/order 25 2024-09-11T08:44:31.951 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdcreate47013 2024-09-11T08:44:31.951 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:31.951 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/order 2024-09-11T08:44:31.951 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=25 2024-09-11T08:44:31.951 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdcreate47013 2024-09-11T08:44:31.951 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/order 2024-09-11T08:44:31.994 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:31.994+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=25 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 25 = 25 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdcreate47013 'journal info' //journal/splay_width 6 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdcreate47013 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/splay_width 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=6 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdcreate47013 2024-09-11T08:44:32.006 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-09-11T08:44:32.064 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=6 2024-09-11T08:44:32.064 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 6 = 6 2024-09-11T08:44:32.064 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdcreate47013 'journal info' //journal/object_pool rbd 2024-09-11T08:44:32.064 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdcreate47013 2024-09-11T08:44:32.064 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:32.064 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/object_pool 2024-09-11T08:44:32.065 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=rbd 2024-09-11T08:44:32.065 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdcreate47013 2024-09-11T08:44:32.065 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-09-11T08:44:32.068 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:32.068 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:32.075 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:32.075+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:32.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:32.117 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=rbd 2024-09-11T08:44:32.117 INFO:tasks.workunit.client.0.smithi045.stderr:+ test rbd = rbd 2024-09-11T08:44:32.117 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdcreate47013 2024-09-11T08:44:32.176 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:32.176+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:32.196 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image2024-09-11T08:44:32.195+0000 7f8a0ec3e640 0 -- 172.21.15.45:0/3828712341 >> [v2:172.21.15.86:6824/498386186,v1:172.21.15.86:6825/498386186] conn(0x55f0f1ff4fe0 msgr2=0x55f0f1ff7470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:32.199 INFO:tasks.workunit.client.0.smithi045.stderr:: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete...2024-09-11T08:44:32.199+0000 7f8a0643d640 0 -- 172.21.15.45:0/3828712341 >> [v2:172.21.15.86:6816/2975777507,v1:172.21.15.86:6817/2975777507] conn(0x7f89e8069cc0 msgr2=0x7f89e806a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-11T08:44:32.238 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:32.238 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:32.248 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:32.250 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-09-11T08:44:32.254 INFO:tasks.workunit.client.0.smithi045.stderr:+ set +x 2024-09-11T08:44:32.254 INFO:tasks.workunit.client.0.smithi045.stderr:+ test_rbd_copy 2024-09-11T08:44:32.254 INFO:tasks.workunit.client.0.smithi045.stderr:+ local src=testrbdcopys47013 2024-09-11T08:44:32.254 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd create --size 256 testrbdcopys47013 2024-09-11T08:44:32.277 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:32.278+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:32.304 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdcopy47013 2024-09-11T08:44:32.304 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd copy --image-feature exclusive-lock --image-feature journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 testrbdcopys47013 testrbdcopy47013 2024-09-11T08:44:32.368 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:32.368 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:32.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:32.378 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:32.378+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:32.397 INFO:tasks.workunit.client.0.smithi045.stderr: Image copy: 1% complete... Image copy: 3% complete... Image copy: 4% complete... Image copy: 6% complete... Image copy: 7% complete... Image copy: 9% complete... Image copy: 10% complete... Image copy: 12% complete... Image copy: 14% complete... Image copy: 15% complete... Image copy: 17% complete... Image copy: 18% complete... Image copy: 20% complete... Image copy: 21% complete... Image copy: 23% complete... Image copy: 25% complete... Image copy: 26% complete... Image copy: 28% complete... Image copy: 29% complete... Image copy: 31% complete... Image copy: 32% complete... Image copy: 34% complete... Image copy: 35% complete... Image copy: 37% complete... Image copy: 39% complete... Image copy: 40% complete... Image copy: 42% complete... Image copy: 43% complete... Image copy: 45% complete... Image copy: 46% complete... Image copy: 48% complete... Image copy: 50% complete... Image copy: 51% complete... Image copy: 53% complete... Image copy: 54% complete... Image copy: 56% complete... Image copy: 57% complete... Image copy: 59% complete... Image copy: 60% complete... Image copy: 62% complete... Image copy: 64% complete... Image copy: 65% complete... Image copy: 67% complete... Image copy: 68% complete... Image copy: 70% complete... Image copy: 71% complete... Image copy: 73% complete... Image copy: 75% complete... Image copy: 76% complete...2024-09-11T08:44:32.397+0000 7f54c3a44640 0 -- 172.21.15.45:0/2082885993 >> [v2:172.21.15.86:6808/1261869816,v1:172.21.15.86:6809/1261869816] conn(0x7f54a0027390 msgr2=0x7f54a0027f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:32.405 INFO:tasks.workunit.client.0.smithi045.stderr: Image copy: 78% complete... Image copy: 79% complete... Image copy: 81% complete... Image copy: 82% complete... Image copy: 84% complete... Image copy: 85% complete... Image copy: 87% complete... Image copy: 89% complete... Image copy: 90% complete... Image copy: 92% complete... Image copy: 93% complete... Image copy: 95% complete... Image copy: 96% complete... Image copy: 98% complete... Image copy: 100% complete...2024-09-11T08:44:32.405+0000 7f54c3243640 0 -- 172.21.15.45:0/2082885993 >> [v2:172.21.15.86:6824/498386186,v1:172.21.15.86:6825/498386186] conn(0x7f54a4004a70 msgr2=0x7f54a40074b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:32.406 INFO:tasks.workunit.client.0.smithi045.stderr: Image copy: 100% complete...done. 2024-09-11T08:44:32.410 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdcopys47013 2024-09-11T08:44:32.479 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:32.479+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:32.487 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:32.487 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:32.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:32.507 INFO:tasks.workunit.client.0.smithi045.stderr:2024-09-11T08:44:32.504+0000 7feb820aa640 0 -- 172.21.15.45:0/3257741465 >> [v2:172.21.15.45:6824/1040844445,v1:172.21.15.45:6825/1040844445] conn(0x55b7dbf9f8f0 msgr2=0x55b7dbfa1ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:32.518 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-09-11T08:44:32.521 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdcopy47013 'journal info' //journal/order 25 2024-09-11T08:44:32.522 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdcopy47013 2024-09-11T08:44:32.522 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:32.522 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/order 2024-09-11T08:44:32.522 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=25 2024-09-11T08:44:32.522 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdcopy47013 2024-09-11T08:44:32.522 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/order 2024-09-11T08:44:32.574 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=25 2024-09-11T08:44:32.574 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 25 = 25 2024-09-11T08:44:32.574 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdcopy47013 'journal info' //journal/splay_width 6 2024-09-11T08:44:32.575 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdcopy47013 2024-09-11T08:44:32.575 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:32.575 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/splay_width 2024-09-11T08:44:32.575 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=6 2024-09-11T08:44:32.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdcopy47013 2024-09-11T08:44:32.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-09-11T08:44:32.581 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:32.582+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:32.627 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=6 2024-09-11T08:44:32.627 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 6 = 6 2024-09-11T08:44:32.627 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdcopy47013 'journal info' //journal/object_pool rbd 2024-09-11T08:44:32.627 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdcopy47013 2024-09-11T08:44:32.627 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:32.627 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/object_pool 2024-09-11T08:44:32.628 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=rbd 2024-09-11T08:44:32.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdcopy47013 2024-09-11T08:44:32.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-09-11T08:44:32.642 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:32.642 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:32.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:32.654 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 7, 1, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:44:32.655 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:44:32.655 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T08:44:32.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T08:44:32.681 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=rbd 2024-09-11T08:44:32.681 INFO:tasks.workunit.client.0.smithi045.stderr:+ test rbd = rbd 2024-09-11T08:44:32.681 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdcopy47013 2024-09-11T08:44:32.682 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:32.681+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:32.761 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... 2024-09-11T08:44:32.759+0000 7f271d615640 0 -- 172.21.15.45:0/1938741253 >> [v2:172.21.15.45:6824/1040844445,v1:172.21.15.45:6825/1040844445] conn(0x5595922cef00 msgr2=0x5595922d8a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:32.768 INFO:tasks.workunit.client.0.smithi045.stderr:Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete...2024-09-11T08:44:32.767+0000 7f271c3b3640 0 -- 172.21.15.45:0/1938741253 >> [v2:172.21.15.45:6808/2540486849,v1:172.21.15.45:6809/2540486849] conn(0x7f26fc06f9d0 msgr2=0x7f26fc068660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-11T08:44:32.782 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:32.782+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:32.805 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:32.805 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:32.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:32.883 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:32.882+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 60, 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "75755954-09e8-48d3-b9c1-83899efaff39", 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-11T08:42:57.181838+0000", 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-11T08:44:31.649791+0000", 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "last_up_change": "2024-09-11T08:43:09.343672+0000", 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "last_in_change": "2024-09-11T08:44:27.636062+0000", 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_num": 5804032, 2024-09-11T08:44:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_set": [ 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "nodeep-scrub", 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "pglog_hardlimit", 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "purged_snapdirs", 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_deletes", 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "sortbitwise" 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_version": 5, 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_snapshot": "", 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_max": 2, 2024-09-11T08:44:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd": 8, 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "require_min_compat_client": "luminous", 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min_compat_client": "luminous", 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "require_osd_release": "squid", 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_crimson": false, 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "pools": [ 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 1, 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": ".mgr", 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-11T08:43:08.201248+0000", 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool", 2024-09-11T08:44:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 1, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 1, 2024-09-11T08:44:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 1, 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 1, 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 1, 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "17", 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:44:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 0, 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 0, 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-11T08:44:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-11T08:44:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "options": { 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_max": 32, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_min": 1 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": {} 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 8, 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 8, 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.25, 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 2, 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 2, 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-11T08:44:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 2, 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": "rbd", 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-11T08:43:16.189979+0000", 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 8193, 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-11T08:44:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 8, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 8, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 8, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 8, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 8, 2024-09-11T08:44:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "60", 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:44:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 3, 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 60, 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-11T08:44:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-11T08:44:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "options": {}, 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd": {} 2024-09-11T08:44:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 2, 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 2, 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.75, 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 1.5, 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 1.5, 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "ac6b86cf-4ee8-49eb-a680-a350b1c9879c", 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 49, 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6808", 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6809", 2024-09-11T08:44:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6810", 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6811", 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6814", 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6815", 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6812", 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6813", 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2540486849 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6809/2540486849", 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6811/2540486849", 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6815/2540486849", 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6813/2540486849", 2024-09-11T08:44:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "e27d776b-f77b-4111-a536-5f5d8030c696", 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 57, 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6824", 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6825", 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6826", 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6827", 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6830", 2024-09-11T08:44:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6831", 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6828", 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6829", 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1040844445 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6825/1040844445", 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6827/1040844445", 2024-09-11T08:44:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6831/1040844445", 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6829/1040844445", 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "6b8e44f6-395b-447d-a5d3-c19a0a4be61b", 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 45, 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6816", 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6817", 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6818", 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6819", 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6822", 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6823", 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6820", 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6821", 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1231172466 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6817/1231172466", 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6819/1231172466", 2024-09-11T08:44:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6823/1231172466", 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6821/1231172466", 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "1d1e1983-4c89-4cb4-a6b4-af244f941a7a", 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 13, 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 0, 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6802", 2024-09-11T08:44:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6803", 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6806", 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6807", 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6804", 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6805", 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 226138367 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6801/226138367", 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6803/226138367", 2024-09-11T08:44:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6807/226138367", 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6805/226138367", 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "fd9239ee-3a74-4399-af2f-1e9a63a2f642", 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 13, 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 0, 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6800", 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6801", 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6802", 2024-09-11T08:44:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6803", 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6806", 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6807", 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6804", 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6805", 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 374760845 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6801/374760845", 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6803/374760845", 2024-09-11T08:44:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6807/374760845", 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6805/374760845", 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "cfff2555-5ad4-4629-a725-afadf9b6a96e", 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 34, 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6824", 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6825", 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6826", 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6827", 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6830", 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6831", 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6828", 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6829", 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 498386186 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6825/498386186", 2024-09-11T08:44:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6827/498386186", 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6831/498386186", 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6829/498386186", 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "4588b13e-af90-48a7-a10d-f7a185a902e2", 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 57, 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6808", 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6809", 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6810", 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6811", 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6814", 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6815", 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6812", 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6813", 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1261869816 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6809/1261869816", 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6811/1261869816", 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6815/1261869816", 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6813/1261869816", 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "12da1d21-9c59-4462-9d5a-61801dab3e4d", 2024-09-11T08:44:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 15, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 54, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6816", 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6817", 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6818", 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6819", 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6822", 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6823", 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-11T08:44:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6820", 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.86:6821", 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2975777507 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.86:6817/2975777507", 2024-09-11T08:44:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.86:6819/2975777507", 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2975777507", 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2975777507", 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_xinfo": [ 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.394370+0000", 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.813228+0000", 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.425286+0000", 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.147591+0000", 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:07.119248+0000", 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.653838+0000", 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.459308+0000", 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540701547738038271, 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-11T08:43:06.492589+0000", 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap": [ 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "pgid": "1.0", 2024-09-11T08:44:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: 2, 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: 0 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "pgid": "2.1", 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: 0, 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: 1 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_items": [], 2024-09-11T08:44:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_primaries": [], 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_temp": [], 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_temp": [], 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklist": {}, 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "range_blocklist": {}, 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profiles": { 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default": { 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-failure-domain": "osd", 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "k": "2", 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "m": "1", 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin": "jerasure", 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "technique": "reed_sol_van" 2024-09-11T08:44:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps_queue": [ 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 2, 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "snaps": [ 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "begin": 3, 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "length": 1 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "new_removed_snaps": [ 2024-09-11T08:44:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 2, 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "snaps": [ 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "begin": 3, 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "length": 1 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "new_purged_snaps": [], 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_node_flags": {}, 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "device_class_flags": {}, 2024-09-11T08:44:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": { 2024-09-11T08:44:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_enabled": false, 2024-09-11T08:44:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_bucket_count": 0, 2024-09-11T08:44:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "degraded_stretch_mode": 0, 2024-09-11T08:44:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "recovering_stretch_mode": 0, 2024-09-11T08:44:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_bucket": 0 2024-09-11T08:44:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:33.033 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:33.035 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:33.035 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:33.036 INFO:tasks.thrashosds.thrasher:j is {'epoch': 60, 'fsid': '75755954-09e8-48d3-b9c1-83899efaff39', 'created': '2024-09-11T08:42:57.181838+0000', 'modified': '2024-09-11T08:44:31.649791+0000', 'last_up_change': '2024-09-11T08:43:09.343672+0000', 'last_in_change': '2024-09-11T08:44:27.636062+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:43:08.201248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T08:43:16.189979+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'ac6b86cf-4ee8-49eb-a680-a350b1c9879c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6808', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6809', 'nonce': 2540486849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6810', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6811', 'nonce': 2540486849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6814', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6815', 'nonce': 2540486849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6812', 'nonce': 2540486849}, {'type': 'v1', 'addr': '172.21.15.45:6813', 'nonce': 2540486849}]}, 'public_addr': '172.21.15.45:6809/2540486849', 'cluster_addr': '172.21.15.45:6811/2540486849', 'heartbeat_back_addr': '172.21.15.45:6815/2540486849', 'heartbeat_front_addr': '172.21.15.45:6813/2540486849', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e27d776b-f77b-4111-a536-5f5d8030c696', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6824', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6825', 'nonce': 1040844445}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6826', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6827', 'nonce': 1040844445}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6830', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6831', 'nonce': 1040844445}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6828', 'nonce': 1040844445}, {'type': 'v1', 'addr': '172.21.15.45:6829', 'nonce': 1040844445}]}, 'public_addr': '172.21.15.45:6825/1040844445', 'cluster_addr': '172.21.15.45:6827/1040844445', 'heartbeat_back_addr': '172.21.15.45:6831/1040844445', 'heartbeat_front_addr': '172.21.15.45:6829/1040844445', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '6b8e44f6-395b-447d-a5d3-c19a0a4be61b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6816', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6817', 'nonce': 1231172466}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6818', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6819', 'nonce': 1231172466}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6822', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6823', 'nonce': 1231172466}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6820', 'nonce': 1231172466}, {'type': 'v1', 'addr': '172.21.15.45:6821', 'nonce': 1231172466}]}, 'public_addr': '172.21.15.45:6817/1231172466', 'cluster_addr': '172.21.15.45:6819/1231172466', 'heartbeat_back_addr': '172.21.15.45:6823/1231172466', 'heartbeat_front_addr': '172.21.15.45:6821/1231172466', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1d1e1983-4c89-4cb4-a6b4-af244f941a7a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6800', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6801', 'nonce': 226138367}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6802', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6803', 'nonce': 226138367}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6806', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6807', 'nonce': 226138367}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6804', 'nonce': 226138367}, {'type': 'v1', 'addr': '172.21.15.45:6805', 'nonce': 226138367}]}, 'public_addr': '172.21.15.45:6801/226138367', 'cluster_addr': '172.21.15.45:6803/226138367', 'heartbeat_back_addr': '172.21.15.45:6807/226138367', 'heartbeat_front_addr': '172.21.15.45:6805/226138367', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fd9239ee-3a74-4399-af2f-1e9a63a2f642', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6800', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6801', 'nonce': 374760845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6802', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6803', 'nonce': 374760845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6806', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6807', 'nonce': 374760845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6804', 'nonce': 374760845}, {'type': 'v1', 'addr': '172.21.15.86:6805', 'nonce': 374760845}]}, 'public_addr': '172.21.15.86:6801/374760845', 'cluster_addr': '172.21.15.86:6803/374760845', 'heartbeat_back_addr': '172.21.15.86:6807/374760845', 'heartbeat_front_addr': '172.21.15.86:6805/374760845', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'cfff2555-5ad4-4629-a725-afadf9b6a96e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6824', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6825', 'nonce': 498386186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6826', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6827', 'nonce': 498386186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6830', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6831', 'nonce': 498386186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6828', 'nonce': 498386186}, {'type': 'v1', 'addr': '172.21.15.86:6829', 'nonce': 498386186}]}, 'public_addr': '172.21.15.86:6825/498386186', 'cluster_addr': '172.21.15.86:6827/498386186', 'heartbeat_back_addr': '172.21.15.86:6831/498386186', 'heartbeat_front_addr': '172.21.15.86:6829/498386186', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '4588b13e-af90-48a7-a10d-f7a185a902e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6808', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6809', 'nonce': 1261869816}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6810', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6811', 'nonce': 1261869816}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6814', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6815', 'nonce': 1261869816}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6812', 'nonce': 1261869816}, {'type': 'v1', 'addr': '172.21.15.86:6813', 'nonce': 1261869816}]}, 'public_addr': '172.21.15.86:6809/1261869816', 'cluster_addr': '172.21.15.86:6811/1261869816', 'heartbeat_back_addr': '172.21.15.86:6815/1261869816', 'heartbeat_front_addr': '172.21.15.86:6813/1261869816', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '12da1d21-9c59-4462-9d5a-61801dab3e4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6816', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6817', 'nonce': 2975777507}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6818', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6819', 'nonce': 2975777507}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6822', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6823', 'nonce': 2975777507}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6820', 'nonce': 2975777507}, {'type': 'v1', 'addr': '172.21.15.86:6821', 'nonce': 2975777507}]}, 'public_addr': '172.21.15.86:6817/2975777507', 'cluster_addr': '172.21.15.86:6819/2975777507', 'heartbeat_back_addr': '172.21.15.86:6823/2975777507', 'heartbeat_front_addr': '172.21.15.86:6821/2975777507', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.394370+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.813228+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.425286+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.147591+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:07.119248+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.653838+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.459308+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:43:06.492589+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.0', 'osds': [2, 0]}, {'pgid': '2.1', 'osds': [0, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 2, 'snaps': [{'begin': 3, 'length': 1}]}], 'new_removed_snaps': [{'pool': 2, 'snaps': [{'begin': 3, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T08:44:33.036 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-11T08:44:33.037 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:33.037 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-09-11T08:44:33.037 INFO:tasks.workunit.client.0.smithi045.stderr:+ set +x 2024-09-11T08:44:33.037 INFO:tasks.workunit.client.0.smithi045.stderr:+ test_rbd_clone 2024-09-11T08:44:33.037 INFO:tasks.workunit.client.0.smithi045.stderr:+ local parent=testrbdclonep47013 2024-09-11T08:44:33.038 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd create --image-feature layering --size 256 testrbdclonep47013 2024-09-11T08:44:33.038 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:33.034+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:33.041 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd snap create testrbdclonep47013@snap 2024-09-11T08:44:33.136 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:33.136+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:33.140 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:33.140 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:33.150 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:33.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:33.236 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:33.237+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:33.293 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:33.293 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:33.302 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:33.311 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:33.328 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:33.337 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:33.336+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:33.345 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:33.345 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:33.361 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:33.361 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:33.378 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:33.394 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:33.395 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:33.412 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:33.429 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:33.437 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:33.437+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:33.448 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:33.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:33.538 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:33.538+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:33.559 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:33.560 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:33.560 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:33.560 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:33.560 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:33.569 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:33.639 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:33.640+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:33.658 INFO:tasks.workunit.client.0.smithi045.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-11T08:44:33.663 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd snap protect testrbdclonep47013@snap 2024-09-11T08:44:33.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4914.0:6 2.6 e58/57 2:678275a5:::rbd_id.testrbdjournal47013:head v 58'1, mlcod=0'0)", 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:28.941127+0000", 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.7326583920000003, 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015120800000000001, 2024-09-11T08:44:33.674 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941127+0000", 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941127+0000", 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941128+0000", 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2099999999999995e-07 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941134+0000", 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9349999999999999e-06 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941145+0000", 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0679999999999999e-05 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941152+0000", 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9990000000000002e-06 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941201+0000", 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8789999999999999e-05 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.941224+0000", 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3781000000000002e-05 2024-09-11T08:44:33.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942607+0000", 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013825790000000001 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:28.942639+0000", 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2495e-05 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.5115.0:12 2.6 e58/57 2:60b3d0f7:::journal_data.2.13daa594a2f0.1:head v 58'4, mlcod=58'1)", 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.103162+0000", 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.5706234220000002, 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0040232690000000003, 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103162+0000", 2024-09-11T08:44:33.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103162+0000", 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103163+0000", 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7800000000000001e-07 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103175+0000", 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2257e-05 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103186+0000", 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0910000000000001e-05 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103195+0000", 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7229999999999993e-06 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103236+0000", 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1511000000000003e-05 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103249+0000", 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2840000000000001e-05 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.107144+0000", 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038949800000000001 2024-09-11T08:44:33.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.107186+0000", 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1369999999999999e-05 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.5115.0:16 2.6 e58/57 2:60b3d0f7:::journal_data.2.13daa594a2f0.1:head v 58'5, mlcod=58'1)", 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.103828+0000", 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.5699577580000001, 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042693369999999998, 2024-09-11T08:44:33.683 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103828+0000", 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103828+0000", 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103828+0000", 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4299999999999998e-07 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103839+0000", 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0108999999999999e-05 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103853+0000", 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4780000000000001e-05 2024-09-11T08:44:33.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.103859+0000", 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6370000000000004e-06 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.106038+0000", 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021788910000000001 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.106060+0000", 2024-09-11T08:44:33.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2235999999999999e-05 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.108069+0000", 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020090970000000001 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.108097+0000", 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8144e-05 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4892.0:24 2.6 e58/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 58'8, mlcod=58'6)", 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.161037+0000", 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.5127489000000001, 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001893376, 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.161037+0000", 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.161037+0000", 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.161037+0000", 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5799999999999999e-07 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.161045+0000", 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6750000000000002e-06 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.161055+0000", 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6760000000000002e-06 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.161060+0000", 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8409999999999999e-06 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.161112+0000", 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2599000000000002e-05 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.161135+0000", 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2728e-05 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162888+0000", 2024-09-11T08:44:33.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017526449999999999 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162930+0000", 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2654000000000002e-05 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4892.0:26 2.6 e58/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 58'10, mlcod=58'6)", 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.162056+0000", 2024-09-11T08:44:33.692 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.51172971, 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001761185, 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162056+0000", 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162056+0000", 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162059+0000", 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.202e-06 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162066+0000", 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9290000000000001e-06 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162074+0000", 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7389999999999999e-06 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162079+0000", 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162122+0000", 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2862999999999999e-05 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162133+0000", 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1676e-05 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163795+0000", 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001661825 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163817+0000", 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2045999999999999e-05 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.696 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4892.0:27 2.6 e58/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 58'11, mlcod=58'6)", 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.162437+0000", 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.5113488180000001, 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029801509999999999, 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162437+0000", 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162437+0000", 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162440+0000", 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1389999999999999e-06 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162447+0000", 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0809999999999996e-06 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162454+0000", 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8229999999999997e-06 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162458+0000", 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1350000000000002e-06 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162571+0000", 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011239 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162591+0000", 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0944000000000001e-05 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165376+0000", 2024-09-11T08:44:33.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002784211 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165417+0000", 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1427999999999997e-05 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4892.0:28 2.6 e58/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 58'12, mlcod=58'6)", 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.162956+0000", 2024-09-11T08:44:33.701 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.5108293379999997, 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024903830000000001, 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162956+0000", 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162956+0000", 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162957+0000", 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3000000000000001e-07 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162972+0000", 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5209e-05 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162979+0000", 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.686e-06 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.162986+0000", 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8299999999999998e-06 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163021+0000", 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5317000000000001e-05 2024-09-11T08:44:33.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163038+0000", 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6597999999999999e-05 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165422+0000", 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023841600000000002 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165447+0000", 2024-09-11T08:44:33.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5153000000000001e-05 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4892.0:29 2.6 e58/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 58'13, mlcod=58'6)", 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.163492+0000", 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.510293909, 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019798509999999999, 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163492+0000", 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163492+0000", 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163493+0000", 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6400000000000002e-07 2024-09-11T08:44:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163508+0000", 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5398000000000001e-05 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163518+0000", 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9680000000000001e-06 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163524+0000", 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.224e-06 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163563+0000", 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8747e-05 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.163574+0000", 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.083e-05 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165448+0000", 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001874372 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165472+0000", 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3648000000000001e-05 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4892.0:31 2.6 e58/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 58'15, mlcod=58'9)", 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.164650+0000", 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.5091361179999998, 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016677319999999999, 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.164650+0000", 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.164650+0000", 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.164650+0000", 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3600000000000004e-07 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.164657+0000", 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4049999999999997e-06 2024-09-11T08:44:33.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.164664+0000", 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7149999999999998e-06 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.164668+0000", 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9310000000000001e-06 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.164704+0000", 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6152000000000002e-05 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.164717+0000", 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2189e-05 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.166277+0000", 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015607780000000001 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.166317+0000", 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0025999999999999e-05 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4892.0:33 2.6 e58/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 58'17, mlcod=58'9)", 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:30.165440+0000", 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.5083453159999998, 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016833900000000001, 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165440+0000", 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165440+0000", 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165441+0000", 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8699999999999995e-07 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165449+0000", 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6499999999999996e-06 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165457+0000", 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0800000000000006e-06 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165462+0000", 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7540000000000003e-06 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165514+0000", 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1730000000000001e-05 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.165526+0000", 2024-09-11T08:44:33.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1810999999999999e-05 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.167093+0000", 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015667249999999999 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:30.167124+0000", 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1152999999999998e-05 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:25 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'25, mlcod=59'22)", 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.130196+0000", 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5435896329999998, 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071429090000000002, 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.130196+0000", 2024-09-11T08:44:33.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.130196+0000", 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.130197+0000", 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7000000000000004e-07 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.130205+0000", 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1280000000000008e-06 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.130214+0000", 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8899999999999996e-06 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.130220+0000", 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1410000000000002e-06 2024-09-11T08:44:33.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.130261+0000", 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1195000000000001e-05 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.130275+0000", 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3363999999999999e-05 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.137259+0000", 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069847370000000004 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.137339+0000", 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9783999999999994e-05 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:26 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'26, mlcod=59'22)", 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.131001+0000", 2024-09-11T08:44:33.724 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5427848420000001, 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0095134380000000008, 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131001+0000", 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131001+0000", 2024-09-11T08:44:33.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131002+0000", 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0900000000000001e-07 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131011+0000", 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8689999999999993e-06 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131028+0000", 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7108999999999998e-05 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131037+0000", 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1560000000000001e-06 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131076+0000", 2024-09-11T08:44:33.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9443999999999997e-05 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131089+0000", 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2931999999999999e-05 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140481+0000", 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0093917040000000007 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140514+0000", 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3515e-05 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:27 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'27, mlcod=59'22)", 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.131534+0000", 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5422519810000002, 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0089961039999999996, 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.729 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131534+0000", 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131534+0000", 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131534+0000", 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4200000000000001e-07 2024-09-11T08:44:33.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131543+0000", 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3299999999999999e-06 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131549+0000", 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8900000000000001e-06 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131553+0000", 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9310000000000001e-06 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131587+0000", 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3963e-05 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.131599+0000", 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2041e-05 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140516+0000", 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0089165960000000006 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140530+0000", 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3910999999999999e-05 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:28 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'28, mlcod=59'22)", 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.132081+0000", 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5417052789999999, 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0084662580000000008, 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132081+0000", 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132081+0000", 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132081+0000", 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132086+0000", 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7389999999999999e-06 2024-09-11T08:44:33.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132091+0000", 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.648e-06 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132096+0000", 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1909999999999999e-06 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132129+0000", 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2830999999999998e-05 2024-09-11T08:44:33.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132140+0000", 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1226999999999999e-05 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140533+0000", 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083929439999999994 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140547+0000", 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3351e-05 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:29 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'29, mlcod=59'22)", 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.132648+0000", 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5411377939999999, 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0079180829999999994, 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132648+0000", 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132648+0000", 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132648+0000", 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4400000000000001e-07 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132653+0000", 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5190000000000003e-06 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132658+0000", 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3290000000000001e-06 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.132661+0000", 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-06 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133957+0000", 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012959930000000001 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133986+0000", 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9076000000000001e-05 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140548+0000", 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0065618109999999999 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140566+0000", 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7841000000000001e-05 2024-09-11T08:44:33.742 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:30 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'30, mlcod=59'22)", 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.133269+0000", 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5405164129999998, 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0073398539999999998, 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133269+0000", 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133269+0000", 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133271+0000", 2024-09-11T08:44:33.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4839999999999999e-06 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133313+0000", 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1768999999999999e-05 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133338+0000", 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4946000000000001e-05 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133350+0000", 2024-09-11T08:44:33.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612999999999999e-05 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134293+0000", 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00094307099999999997 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134304+0000", 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1054e-05 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140567+0000", 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062624250000000003 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140609+0000", 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2491999999999999e-05 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:31 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'31, mlcod=59'22)", 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.133834+0000", 2024-09-11T08:44:33.747 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5399522970000001, 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0068063919999999997, 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133834+0000", 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133834+0000", 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133835+0000", 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.677e-06 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133898+0000", 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2954999999999996e-05 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133930+0000", 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1766e-05 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.133947+0000", 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7527e-05 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134511+0000", 2024-09-11T08:44:33.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00056364800000000004 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134522+0000", 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.084e-05 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140612+0000", 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060895819999999996 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140640+0000", 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8396999999999999e-05 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:32 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'32, mlcod=59'22)", 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.134339+0000", 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5394472189999999, 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006395151, 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134339+0000", 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134339+0000", 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134339+0000", 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5000000000000002e-07 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134346+0000", 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8730000000000001e-06 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134361+0000", 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4436999999999999e-05 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134365+0000", 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4839999999999998e-06 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134714+0000", 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00034901000000000002 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134726+0000", 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1977999999999999e-05 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140713+0000", 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0059874250000000002 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140734+0000", 2024-09-11T08:44:33.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0293999999999999e-05 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:33 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'33, mlcod=59'22)", 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.134918+0000", 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.5388674990000002, 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0058394120000000004, 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134918+0000", 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134918+0000", 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134919+0000", 2024-09-11T08:44:33.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.72e-07 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134924+0000", 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0409999999999997e-06 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134930+0000", 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9399999999999999e-06 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.134933+0000", 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4379999999999999e-06 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.137421+0000", 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024882559999999999 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.137455+0000", 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3846000000000001e-05 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140736+0000", 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003280402 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.140758+0000", 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2116999999999998e-05 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4937.0:34 2.6 e59/57 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head v 59'34, mlcod=59'22)", 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:31.135620+0000", 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.538165561, 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054264689999999997, 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.135620+0000", 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.135620+0000", 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.135621+0000", 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.135627+0000", 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1619999999999996e-06 2024-09-11T08:44:33.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.135636+0000", 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1290000000000002e-06 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.135652+0000", 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5968999999999999e-05 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.137789+0000", 2024-09-11T08:44:33.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021366919999999999 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.137800+0000", 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1185e-05 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.141028+0000", 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032280149999999999 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:33.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:31.141047+0000", 2024-09-11T08:44:33.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8740999999999999e-05 2024-09-11T08:44:33.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.766 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:33.766 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:33.766 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:33.767 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdclone47013 2024-09-11T08:44:33.767 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd clone --image-feature layering --image-feature exclusive-lock --image-feature journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 testrbdclonep47013@snap testrbdclone47013 2024-09-11T08:44:33.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:33.768 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:33.767+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:33.803 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdclone47013 'journal info' //journal/order 25 2024-09-11T08:44:33.803 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdclone47013 2024-09-11T08:44:33.803 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:33.803 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/order 2024-09-11T08:44:33.803 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=25 2024-09-11T08:44:33.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdclone47013 2024-09-11T08:44:33.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/order 2024-09-11T08:44:33.868 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:33.867+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:33.911 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:33.912 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:33.912 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-11T08:44:33.912 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:33.920 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:33.968 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:33.968+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:34.023 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:34.023 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-11T08:44:34.023 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-09-11T08:44:34.023 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-09-11T08:44:34.023 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:34.031 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:34.066 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=25 2024-09-11T08:44:34.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 25 = 25 2024-09-11T08:44:34.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdclone47013 'journal info' //journal/splay_width 6 2024-09-11T08:44:34.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdclone47013 2024-09-11T08:44:34.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:34.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/splay_width 2024-09-11T08:44:34.067 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=6 2024-09-11T08:44:34.067 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdclone47013 2024-09-11T08:44:34.067 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-09-11T08:44:34.074 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:34.075+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:34.127 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=6 2024-09-11T08:44:34.127 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 6 = 6 2024-09-11T08:44:34.127 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdclone47013 'journal info' //journal/object_pool rbd 2024-09-11T08:44:34.127 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdclone47013 2024-09-11T08:44:34.127 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:34.127 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/object_pool 2024-09-11T08:44:34.128 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=rbd 2024-09-11T08:44:34.128 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdclone47013 2024-09-11T08:44:34.128 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 84.778100379999998, 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047602849, 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 1 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360133+0000", 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360134+0000", 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360140+0000", 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360149+0000", 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.465e-06 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360164+0000", 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4465e-05 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360225+0000", 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1055000000000004e-05 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:09.360238+0000", 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3236e-05 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406049+0000", 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.045810497 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406223+0000", 2024-09-11T08:44:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000174129 2024-09-11T08:44:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.406371+0000", 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407295+0000", 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092332899999999997 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407667+0000", 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037258499999999998 2024-09-11T08:44:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407706+0000", 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8822999999999997e-05 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.407736+0000", 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060999999999999e-05 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.662135535999994, 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069705619999999996, 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 86 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476098+0000", 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476099+0000", 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8800000000000003e-07 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476102+0000", 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4199999999999999e-06 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476109+0000", 2024-09-11T08:44:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476114+0000", 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0119999999999996e-06 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478160+0000", 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020452949999999999 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478207+0000", 2024-09-11T08:44:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7162e-05 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478281+0000", 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4169000000000001e-05 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479586+0000", 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00130506 2024-09-11T08:44:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483007+0000", 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034210629999999998 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483045+0000", 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7731000000000002e-05 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483069+0000", 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3771e-05 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.662111670000002, 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069904470000000003, 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 87 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476122+0000", 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476201+0000", 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8608000000000004e-05 2024-09-11T08:44:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476210+0000", 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9879999999999993e-06 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476215+0000", 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6019999999999999e-06 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478653+0000", 2024-09-11T08:44:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024371890000000002 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478702+0000", 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.939e-05 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.478777+0000", 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4461000000000004e-05 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479915+0000", 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001138043 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483073+0000", 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031579619999999998 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483095+0000", 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2209999999999999e-05 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483112+0000", 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7589999999999999e-05 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.662005612000002, 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069246619999999998, 2024-09-11T08:44:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476228+0000", 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476229+0000", 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476233+0000", 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476241+0000", 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8849999999999999e-06 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.476249+0000", 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4460000000000001e-06 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479223+0000", 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029739200000000001 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479270+0000", 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6263000000000001e-05 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.479342+0000", 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2311999999999995e-05 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.480201+0000", 2024-09-11T08:44:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085868200000000004 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483115+0000", 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029146900000000002 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483135+0000", 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0160999999999999e-05 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.483153+0000", 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.510247700999997, 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069447900000000002, 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 115 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.627986+0000", 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1399999999999997e-07 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628041+0000", 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4778999999999998e-05 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628046+0000", 2024-09-11T08:44:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7439999999999998e-06 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628049+0000", 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3249999999999999e-06 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633235+0000", 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051863179999999997 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633283+0000", 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7317000000000001e-05 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633359+0000", 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5755999999999998e-05 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634404+0000", 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010458069999999999 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634877+0000", 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047269500000000003 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634904+0000", 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7402e-05 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634931+0000", 2024-09-11T08:44:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6233000000000001e-05 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.510178401999994, 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069527310000000002, 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628055+0000", 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.79e-07 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628058+0000", 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3130000000000001e-06 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628065+0000", 2024-09-11T08:44:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0450000000000003e-06 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628075+0000", 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0000000000000001e-05 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633737+0000", 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056622240000000004 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633786+0000", 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8826999999999999e-05 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.633862+0000", 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6298999999999996e-05 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634636+0000", 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00077402899999999997 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634938+0000", 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030200099999999998 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634979+0000", 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0998000000000003e-05 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635008+0000", 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8716000000000001e-05 2024-09-11T08:44:34.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.510153450000004, 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070050299999999998, 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 117 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628080+0000", 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700000000000003e-07 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628084+0000", 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628087+0000", 2024-09-11T08:44:34.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.628091+0000", 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-06 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634118+0000", 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060267569999999998 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634164+0000", 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6819e-05 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634239+0000", 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4902999999999998e-05 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.634789+0000", 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054944100000000001 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635016+0000", 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022711300000000001 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635056+0000", 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9842000000000002e-05 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:10.635085+0000", 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9369999999999998e-05 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.183 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.100456313999999, 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067325429999999997, 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 134 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037777+0000", 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037778+0000", 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037822+0000", 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3886e-05 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037838+0000", 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6710000000000001e-05 2024-09-11T08:44:34.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037851+0000", 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.221e-05 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040087+0000", 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022360050000000001 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040167+0000", 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0391999999999999e-05 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040292+0000", 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124676 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042283+0000", 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019908830000000001 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044458+0000", 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021755530000000002 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044486+0000", 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7727000000000001e-05 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044510+0000", 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3886000000000001e-05 2024-09-11T08:44:34.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.100368700999994, 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066935240000000002, 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 135 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037865+0000", 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037869+0000", 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0130000000000004e-06 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037877+0000", 2024-09-11T08:44:34.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4089999999999997e-06 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037882+0000", 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2340000000000002e-06 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040749+0000", 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028671 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040794+0000", 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4777999999999997e-05 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.040871+0000", 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7169000000000006e-05 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042336+0000", 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014645960000000001 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044514+0000", 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021785870000000001 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044536+0000", 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2200000000000001e-05 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044558+0000", 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.19e-05 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.100329192999993, 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006715057, 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 136 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037904+0000", 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037905+0000", 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.63e-07 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037909+0000", 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8060000000000001e-06 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037917+0000", 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1529999999999997e-06 2024-09-11T08:44:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037922+0000", 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8300000000000003e-06 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041237+0000", 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033158279999999998 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041281+0000", 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3810999999999998e-05 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041349+0000", 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8002000000000004e-05 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042617+0000", 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001267425 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044562+0000", 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001944901 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044601+0000", 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9477999999999997e-05 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044619+0000", 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.836e-05 2024-09-11T08:44:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.100302987999996, 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082441809999999997, 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037931+0000", 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037968+0000", 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6399000000000002e-05 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037972+0000", 2024-09-11T08:44:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037976+0000", 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2090000000000001e-06 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041577+0000", 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003601333 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041654+0000", 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6614999999999997e-05 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.041769+0000", 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115488 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043521+0000", 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017514729999999999 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046100+0000", 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002579479 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046140+0000", 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9836999999999998e-05 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046175+0000", 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4910999999999998e-05 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.100251732999993, 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082827500000000002, 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037982+0000", 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-07 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037984+0000", 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.153e-06 2024-09-11T08:44:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037987+0000", 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1690000000000001e-06 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037990+0000", 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7480000000000002e-06 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042650+0000", 2024-09-11T08:44:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046597610000000001 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042721+0000", 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0687999999999998e-05 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.042840+0000", 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011893200000000001 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044423+0000", 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001583409 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046185+0000", 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017624159999999999 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046222+0000", 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6974000000000002e-05 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046265+0000", 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2172999999999998e-05 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.100239814000005, 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083221739999999999, 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 139 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037994+0000", 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.037996+0000", 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.855e-06 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038001+0000", 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038003+0000", 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.599e-06 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043154+0000", 2024-09-11T08:44:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005151096 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043199+0000", 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5009000000000002e-05 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043270+0000", 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0717999999999996e-05 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044451+0000", 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011807409999999999 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046271+0000", 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018205210000000001 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046298+0000", 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7131000000000002e-05 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046316+0000", 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.100225937000005, 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085223899999999995, 2024-09-11T08:44:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 140 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038008+0000", 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-07 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038036+0000", 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7894000000000001e-05 2024-09-11T08:44:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038041+0000", 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0860000000000001e-06 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038044+0000", 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.393e-06 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043555+0000", 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055112490000000002 2024-09-11T08:44:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043600+0000", 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4762000000000003e-05 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.043670+0000", 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9818000000000001e-05 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044655+0000", 2024-09-11T08:44:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098460999999999991 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046481+0000", 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018268399999999999 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046511+0000", 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9901000000000001e-05 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046530+0000", 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618999999999999e-05 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4222.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.100184394999999, 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085224859999999993, 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4222", 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.86:0/1736095225", 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 141 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038049+0000", 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038052+0000", 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3199999999999998e-06 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038055+0000", 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.095e-06 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.038057+0000", 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6120000000000001e-06 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044080+0000", 2024-09-11T08:44:34.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060229530000000002 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044125+0000", 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4320000000000003e-05 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044194+0000", 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9675e-05 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.044919+0000", 2024-09-11T08:44:34.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00072422600000000001 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046535+0000", 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016163270000000001 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046556+0000", 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0920999999999998e-05 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:43:11.046572+0000", 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5756e-05 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:51 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 20480~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586049+0000", 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5521843369999999, 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075292390000000001, 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 51 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586049+0000", 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586049+0000", 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586050+0000", 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5999999999999999e-07 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586053+0000", 2024-09-11T08:44:34.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2059999999999999e-06 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586057+0000", 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7900000000000001e-06 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586059+0000", 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7429999999999998e-06 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.587728+0000", 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001668262 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.587809+0000", 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1908000000000003e-05 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.587944+0000", 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013496100000000001 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589343+0000", 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013987299999999999 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593521+0000", 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004178106 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593555+0000", 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3312999999999998e-05 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593578+0000", 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3859999999999999e-05 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:52 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 24576~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586121+0000", 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5521127699999999, 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075068119999999999, 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.238 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 52 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586121+0000", 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586121+0000", 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586121+0000", 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7899999999999999e-07 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586125+0000", 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7639999999999999e-06 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586130+0000", 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6750000000000001e-06 2024-09-11T08:44:34.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586139+0000", 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0820000000000005e-06 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588358+0000", 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022190539999999998 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588434+0000", 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6458000000000006e-05 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588561+0000", 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012707699999999999 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589387+0000", 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00082586699999999999 2024-09-11T08:44:34.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593585+0000", 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041978629999999996 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593609+0000", 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3903000000000001e-05 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593628+0000", 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8689999999999999e-05 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:53 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 28672~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586168+0000", 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5520653429999998, 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075023590000000001, 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.244 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 53 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586168+0000", 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586168+0000", 2024-09-11T08:44:34.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586169+0000", 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5100000000000001e-07 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586173+0000", 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1489999999999996e-06 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586177+0000", 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4530000000000004e-06 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586180+0000", 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9210000000000001e-06 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588909+0000", 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002728744 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.588957+0000", 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7728000000000001e-05 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589034+0000", 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7041999999999999e-05 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.590298+0000", 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012642829999999999 2024-09-11T08:44:34.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593631+0000", 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003333005 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593654+0000", 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2668e-05 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593671+0000", 2024-09-11T08:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7014999999999999e-05 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:54 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 32768~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586185+0000", 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5520483990000002, 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075261379999999999, 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.250 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586185+0000", 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586185+0000", 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586185+0000", 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4499999999999998e-07 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586189+0000", 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4520000000000002e-06 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586192+0000", 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2310000000000001e-06 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586196+0000", 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4790000000000001e-06 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589421+0000", 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032254060000000001 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589499+0000", 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7651000000000006e-05 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589633+0000", 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013448099999999999 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.590472+0000", 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00083839899999999998 2024-09-11T08:44:34.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593673+0000", 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00320193 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593695+0000", 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1557e-05 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593711+0000", 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6307000000000002e-05 2024-09-11T08:44:34.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.5028.0:55 2.5 2:a5adf036:::rbd_data.1332e735c7c0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 36864~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-09-11T08:44:29.586199+0000", 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5520345209999995, 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075524069999999997, 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.256 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.5028", 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 55 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586199+0000", 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-09-11T08:44:34.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586199+0000", 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586199+0000", 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2100000000000001e-07 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586203+0000", 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9790000000000004e-06 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586207+0000", 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3919999999999999e-06 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.586209+0000", 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5469999999999999e-06 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589949+0000", 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037401380000000001 2024-09-11T08:44:34.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.589993+0000", 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3665000000000003e-05 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.590065+0000", 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1649000000000005e-05 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.590784+0000", 2024-09-11T08:44:34.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00071953399999999997 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593715+0000", 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029311250000000001 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593736+0000", 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0669000000000001e-05 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-09-11T08:44:34.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-09-11T08:44:29.593751+0000", 2024-09-11T08:44:34.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5488000000000001e-05 2024-09-11T08:44:34.262 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.262 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.262 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.262 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-11T08:44:34.262 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-11T08:44:34.262 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-11T08:44:34.264 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=rbd 2024-09-11T08:44:34.264 INFO:tasks.workunit.client.0.smithi045.stderr:+ test rbd = rbd 2024-09-11T08:44:34.264 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdclone47013 2024-09-11T08:44:34.264 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:34.263+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:34.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T08:44:34.267 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete...2024-09-11T08:44:34.263+0000 7f75eea68640 0 -- 172.21.15.45:0/813409098 >> [v2:172.21.15.45:6808/2540486849,v1:172.21.15.45:6809/2540486849] conn(0x7f75cc0162f0 msgr2=0x7f75cc016770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-11T08:44:34.277 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 2024-09-11T08:44:34.276+0000 7f75eea68640 0 -- 172.21.15.45:0/813409098 >> [v2:172.21.15.86:6824/498386186,v1:172.21.15.86:6825/498386186] conn(0x55a2031020e0 msgr2=0x55a203104570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:34.364 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:34.363+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:34.377 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:34.378 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:34.378 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:34.378 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:34.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T08:44:34.470 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:34.470+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:34.493 INFO:tasks.workunit.client.0.smithi045.stderr:51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-09-11T08:44:34.497 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd snap unprotect testrbdclonep47013@snap 2024-09-11T08:44:34.542 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:34.543 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:34.543 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:34.543 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:34.543 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:34.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T08:44:34.593 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:34.592+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-11T08:44:34.666 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:34.667+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:34.703 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.268194578000006, 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014967730000000001, 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435139+0000", 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435140+0000", 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7899999999999997e-07 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435213+0000", 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2968999999999999e-05 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435226+0000", 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.234e-05 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435231+0000", 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2519999999999999e-06 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435287+0000", 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5928999999999999e-05 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.435308+0000", 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1617e-05 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436595+0000", 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012862399999999999 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436636+0000", 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1547e-05 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'15)", 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.267130414999997, 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001406518, 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.709 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436204+0000", 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2200000000000003e-07 2024-09-11T08:44:34.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436211+0000", 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436223+0000", 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1199000000000001e-05 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436230+0000", 2024-09-11T08:44:34.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7449999999999995e-06 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436281+0000", 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0816999999999998e-05 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.436298+0000", 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6928000000000001e-05 2024-09-11T08:44:34.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437566+0000", 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012679150000000001 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.437610+0000", 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4115e-05 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4222.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.239406849000005, 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015282679999999999, 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463927+0000", 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463928+0000", 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.78e-07 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463941+0000", 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3029e-05 2024-09-11T08:44:34.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.716 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.716 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463953+0000", 2024-09-11T08:44:34.716 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1771e-05 2024-09-11T08:44:34.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.716 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.463959+0000", 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464008+0000", 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8659999999999998e-05 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.464028+0000", 2024-09-11T08:44:34.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0151000000000001e-05 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465419+0000", 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013903570000000001 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:43:10.465455+0000", 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:38 2.5 e58/18 2:acb3f4d3:::journal_data.2.1332e735c7c0.1:head v 58'3, mlcod=58'1)", 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.582839+0000", 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.1204946470000001, 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001582017, 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582839+0000", 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582839+0000", 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582840+0000", 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8599999999999998e-07 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582851+0000", 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1270999999999999e-05 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582860+0000", 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1379999999999996e-06 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582866+0000", 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0109999999999997e-06 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582926+0000", 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9422999999999997e-05 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582946+0000", 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0404e-05 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584382+0000", 2024-09-11T08:44:34.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00143597 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584421+0000", 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9113999999999999e-05 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:40 2.5 e58/18 2:b55ab9c6:::journal_data.2.1332e735c7c0.3:head v 58'4, mlcod=58'1)", 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.583367+0000", 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.1199668249999997, 2024-09-11T08:44:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001571233, 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583367+0000", 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583367+0000", 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583368+0000", 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9100000000000004e-07 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583389+0000", 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1639000000000001e-05 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583396+0000", 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0369999999999997e-06 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583400+0000", 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9160000000000003e-06 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583449+0000", 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9159999999999997e-05 2024-09-11T08:44:34.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583472+0000", 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2376000000000001e-05 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584862+0000", 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001390358 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584938+0000", 2024-09-11T08:44:34.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6155999999999994e-05 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5028.0:42 2.5 e58/18 2:acb3f4d3:::journal_data.2.1332e735c7c0.1:head v 58'5, mlcod=58'1)", 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.583728+0000", 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.1196062260000001, 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00137009, 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583728+0000", 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583728+0000", 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583728+0000", 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6100000000000001e-07 2024-09-11T08:44:34.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583782+0000", 2024-09-11T08:44:34.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4169999999999998e-05 2024-09-11T08:44:34.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583798+0000", 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5838e-05 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583803+0000", 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.763e-06 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583855+0000", 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2157000000000002e-05 2024-09-11T08:44:34.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583873+0000", 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8170999999999999e-05 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585066+0000", 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00119308 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585098+0000", 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1350000000000003e-05 2024-09-11T08:44:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5115.0:10 2.1 2:9f7ad37c:::journal_data.2.13daa594a2f0.3:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.101343+0000", 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.601991269, 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021259709999999999, 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5115", 2024-09-11T08:44:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2941479348", 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 10 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101343+0000", 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101343+0000", 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101344+0000", 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7999999999999993e-07 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101353+0000", 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.884e-06 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101366+0000", 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3040999999999999e-05 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101371+0000", 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.468e-06 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101414+0000", 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2929999999999997e-05 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101526+0000", 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112456 2024-09-11T08:44:34.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101674+0000", 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014795800000000001 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102713+0000", 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010386760000000001 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103347+0000", 2024-09-11T08:44:34.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063360999999999995 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103423+0000", 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6784000000000002e-05 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103469+0000", 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5184e-05 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5115.0:14 2.1 2:9f7ad37c:::journal_data.2.13daa594a2f0.3:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.101776+0000", 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6015582909999999, 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002548827, 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5115", 2024-09-11T08:44:34.739 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2941479348", 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101776+0000", 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101776+0000", 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101777+0000", 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0100000000000003e-07 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101798+0000", 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1659999999999999e-05 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101811+0000", 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2510999999999999e-05 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101819+0000", 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999999e-06 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102091+0000", 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027193200000000001 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102173+0000", 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2126999999999995e-05 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102356+0000", 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018356399999999999 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103503+0000", 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001146509 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104260+0000", 2024-09-11T08:44:34.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075766199999999998 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104289+0000", 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8639000000000001e-05 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104324+0000", 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5352000000000003e-05 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.5115.0:9 2.5 e58/18 2:a93e661c:::journal_data.2.13daa594a2f0.2:head v 58'18, mlcod=58'16)", 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.102487+0000", 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.60084698, 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001490618, 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102487+0000", 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102487+0000", 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102488+0000", 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6499999999999998e-07 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102497+0000", 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3929999999999997e-06 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102511+0000", 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3808999999999999e-05 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102518+0000", 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4189999999999996e-06 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102592+0000", 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4028000000000003e-05 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102611+0000", 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8383e-05 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103934+0000", 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013234539999999999 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103978+0000", 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3266999999999998e-05 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5115.0:18 2.1 2:9f7ad37c:::journal_data.2.13daa594a2f0.3:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.102799+0000", 2024-09-11T08:44:34.749 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6005344790000002, 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001838073, 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5115", 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2941479348", 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 18 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102799+0000", 2024-09-11T08:44:34.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102799+0000", 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102800+0000", 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0419999999999999e-06 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102806+0000", 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8889999999999999e-06 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102818+0000", 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1263e-05 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102829+0000", 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1119e-05 2024-09-11T08:44:34.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102877+0000", 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8047000000000003e-05 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102934+0000", 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7142000000000003e-05 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103027+0000", 2024-09-11T08:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3517999999999995e-05 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103962+0000", 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093438899999999999 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104576+0000", 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061375999999999998 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104614+0000", 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8485999999999997e-05 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104638+0000", 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3417999999999998e-05 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5142.0:1 2.1 2:9eb3d535:::journal.13daa594a2f0:head [call journal.client_register in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-11T08:44:34.755 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.764983+0000", 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.9383504509999998, 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012467159999999999, 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5142", 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3942304305", 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.764983+0000", 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.764983+0000", 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.764985+0000", 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.626e-06 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.764991+0000", 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6930000000000004e-06 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765000+0000", 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6099999999999995e-06 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765009+0000", 2024-09-11T08:44:34.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1070000000000005e-06 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765075+0000", 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6674999999999994e-05 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765141+0000", 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5778999999999995e-05 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:34.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765328+0000", 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000186906 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.765901+0000", 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000572983 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.766173+0000", 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027190800000000001 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.766202+0000", 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9564999999999999e-05 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.766230+0000", 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7864e-05 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5142.0:13 2.1 2:9eb3d535:::journal.13daa594a2f0:head [call journal.client_unregister in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.770592+0000", 2024-09-11T08:44:34.761 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.9327414799999998, 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013752809999999999, 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5142", 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3942304305", 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 13 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770592+0000", 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770592+0000", 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770593+0000", 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2500000000000005e-07 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770599+0000", 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7649999999999999e-06 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770610+0000", 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1447999999999999e-05 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770615+0000", 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9529999999999998e-06 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770674+0000", 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8573000000000003e-05 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.770761+0000", 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7198000000000001e-05 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771034+0000", 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027312000000000001 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771567+0000", 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00053294499999999995 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771916+0000", 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034918599999999997 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771946+0000", 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9335999999999998e-05 2024-09-11T08:44:34.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.771968+0000", 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2031999999999998e-05 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5346.0:16 2.1 2:8c0a46ac:::rbd_header.14d94f864dcf:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:33.655353+0000", 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.0479808159999999, 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035938960000000001, 2024-09-11T08:44:34.767 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5346", 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2592511139", 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 16 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.655353+0000", 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.655353+0000", 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.655355+0000", 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.655367+0000", 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.189e-05 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.655388+0000", 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1202000000000001e-05 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.655398+0000", 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9839999999999996e-06 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.655462+0000", 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3706000000000003e-05 2024-09-11T08:44:34.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.655467+0000", 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0050000000000004e-06 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.657223+0000", 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001756128 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.657328+0000", 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000104734 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.657743+0000", 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00041470899999999998 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.658493+0000", 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075067500000000002 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:34.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.658839+0000", 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034555700000000003 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.658898+0000", 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8981999999999999e-05 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:33.658947+0000", 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9224999999999997e-05 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.773 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:74 2.1 2:9875a487:::rbd_data.14f76f1ec7d3.0000000000000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.466304+0000", 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.237030146, 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018664, 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 74 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466304+0000", 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466304+0000", 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466305+0000", 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.17e-06 2024-09-11T08:44:34.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466310+0000", 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9760000000000003e-06 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466319+0000", 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4210000000000001e-06 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466329+0000", 2024-09-11T08:44:34.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0112000000000001e-05 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466390+0000", 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0547000000000002e-05 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466491+0000", 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101178 2024-09-11T08:44:34.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.468170+0000", 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016789960000000001 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:78 2.1 2:84310a8a:::rbd_data.14f76f1ec7d3.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.466351+0000", 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.23698291799999999, 2024-09-11T08:44:34.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020363199999999999, 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 78 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466351+0000", 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466351+0000", 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466352+0000", 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9999999999999998e-07 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466354+0000", 2024-09-11T08:44:34.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5959999999999998e-06 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466362+0000", 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0830000000000004e-06 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466368+0000", 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7300000000000002e-06 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466744+0000", 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00037570300000000002 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466898+0000", 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000154599 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.468387+0000", 2024-09-11T08:44:34.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001489109 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:80 2.1 2:989ccc01:::rbd_data.14f76f1ec7d3.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.466392+0000", 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.23694213, 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020786089999999999, 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:34.783 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 80 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466392+0000", 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466392+0000", 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466392+0000", 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4799999999999998e-07 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466396+0000", 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.749e-06 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466404+0000", 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7249999999999997e-06 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466410+0000", 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1399999999999997e-06 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.467202+0000", 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079184600000000002 2024-09-11T08:44:34.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.467330+0000", 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000127704 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.468470+0000", 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011408970000000001 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:91 2.1 2:9dc75303:::rbd_data.14f76f1ec7d3.0000000000000014:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.466419+0000", 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.23691511800000001, 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028507879999999999, 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 91 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466419+0000", 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466419+0000", 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466419+0000", 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7800000000000002e-07 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466424+0000", 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.181e-06 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466431+0000", 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9800000000000001e-06 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466438+0000", 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.545e-06 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.467619+0000", 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011806060000000001 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.467745+0000", 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000126427 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.469270+0000", 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001524571 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:96 2.1 2:93fa9d1d:::rbd_data.14f76f1ec7d3.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.466448+0000", 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.23688641999999999, 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029788670000000001, 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 96 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466448+0000", 2024-09-11T08:44:34.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466448+0000", 2024-09-11T08:44:34.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466448+0000", 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8800000000000002e-07 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466450+0000", 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3929999999999998e-06 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466457+0000", 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7989999999999996e-06 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466463+0000", 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3299999999999998e-06 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.468515+0000", 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020519309999999999 2024-09-11T08:44:34.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.468585+0000", 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0214000000000002e-05 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.469426+0000", 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00084161200000000005 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.797 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:133 2.1 2:9a7e5247:::rbd_data.14f76f1ec7d3.000000000000003e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.466470+0000", 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.236863935, 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031685760000000002, 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 133 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466470+0000", 2024-09-11T08:44:34.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466470+0000", 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466470+0000", 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9200000000000002e-07 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466474+0000", 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.224e-06 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466480+0000", 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.466485+0000", 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8169999999999998e-06 2024-09-11T08:44:34.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.468765+0000", 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022798290000000001 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.468866+0000", 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000100506 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.469639+0000", 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00077304800000000003 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:150 2.1 2:94cdaf7a:::journal_data.2.14f76f1ec7d3.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+full_force+supports_pool_eio e61)", 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.482767+0000", 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.220566966, 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014710680000000001, 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:34.802 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 150 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.482767+0000", 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.482767+0000", 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.482768+0000", 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0699999999999999e-06 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.482773+0000", 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7319999999999998e-06 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.482783+0000", 2024-09-11T08:44:34.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0069e-05 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.482796+0000", 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3388e-05 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.482848+0000", 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1901999999999999e-05 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:34.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.482936+0000", 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8043999999999999e-05 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.484238+0000", 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013018629999999999 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:34.806 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:34.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T08:44:34.808 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd snap purge testrbdclonep47013 2024-09-11T08:44:34.809 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:34.809+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:34.908 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:34.908+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:34.956 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:34.956 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:34.957 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:34.957 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:34.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T08:44:35.009 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:35.009+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:35.110 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:35.111+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:35.111 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:35.111 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:35.111 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-11T08:44:35.111 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-11T08:44:35.111 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:35.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T08:44:35.210 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:35.210+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:35.229 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:35.229 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-11T08:44:35.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-11T08:44:35.229 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-11T08:44:35.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.229 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5028.0:41 2.4 2:2460a7a9:::journal_data.2.1332e735c7c0.0:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.582514+0000", 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.6458278479999997, 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027589730000000001, 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5028", 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 41 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582514+0000", 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582514+0000", 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582515+0000", 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5000000000000002e-07 2024-09-11T08:44:35.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582523+0000", 2024-09-11T08:44:35.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5000000000000002e-06 2024-09-11T08:44:35.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582532+0000", 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4709999999999996e-06 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582539+0000", 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8170000000000001e-06 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582980+0000", 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044164200000000001 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583029+0000", 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8405000000000002e-05 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.583117+0000", 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.81e-05 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585107+0000", 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019901490000000001 2024-09-11T08:44:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585163+0000", 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5748000000000002e-05 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585230+0000", 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7189999999999999e-05 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585273+0000", 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3300999999999998e-05 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5028.0:44 2.4 2:2982b1d3:::journal_data.2.1332e735c7c0.2:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:29.582896+0000", 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.6454462410000001, 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027890089999999998, 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.236 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5028", 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/1181699499", 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 44 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582896+0000", 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582896+0000", 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582897+0000", 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.99e-07 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582913+0000", 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5942000000000001e-05 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582921+0000", 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2470000000000008e-06 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.582927+0000", 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.879e-06 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584210+0000", 2024-09-11T08:44:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012834809999999999 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584279+0000", 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8442999999999995e-05 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.584455+0000", 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017676500000000001 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585459+0000", 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010038860000000001 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585593+0000", 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000134086 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585646+0000", 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2521000000000001e-05 2024-09-11T08:44:35.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:29.585685+0000", 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9060000000000002e-05 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5115.0:12 2.6 2:60b3d0f7:::journal_data.2.13daa594a2f0.1:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.101684+0000", 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.1266580509999997, 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0055950150000000001, 2024-09-11T08:44:35.242 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5115", 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2941479348", 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 12 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101684+0000", 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101684+0000", 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101685+0000", 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2200000000000004e-07 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101696+0000", 2024-09-11T08:44:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1426e-05 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101702+0000", 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4229999999999996e-06 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.101708+0000", 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2149999999999998e-06 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102744+0000", 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010361070000000001 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102819+0000", 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.47e-05 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102983+0000", 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000164606 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104187+0000", 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012034050000000001 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.107225+0000", 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030379489999999999 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.107254+0000", 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9468e-05 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.107279+0000", 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5194e-05 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5115.0:16 2.6 2:60b3d0f7:::journal_data.2.13daa594a2f0.1:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.102253+0000", 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.1260887290000001, 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059256170000000002, 2024-09-11T08:44:35.248 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5115", 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2941479348", 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 16 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102253+0000", 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102253+0000", 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102255+0000", 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0720000000000001e-06 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102263+0000", 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3580000000000003e-06 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102274+0000", 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1477999999999999e-05 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.102282+0000", 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6510000000000001e-06 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103419+0000", 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001137094 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103511+0000", 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1860000000000005e-05 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.103683+0000", 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000172466 2024-09-11T08:44:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.104469+0000", 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078541500000000003 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.108131+0000", 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036623229999999999 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.108158+0000", 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7047999999999999e-05 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.108179+0000", 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0852e-05 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:24 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 0~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.254 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160657+0000", 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.067685515, 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030238639999999998, 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 24 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160657+0000", 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160657+0000", 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160657+0000", 2024-09-11T08:44:35.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5799999999999998e-07 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160664+0000", 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3899999999999998e-06 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160674+0000", 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0101e-05 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160681+0000", 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3069999999999997e-06 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160724+0000", 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2830000000000002e-05 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160813+0000", 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8961000000000004e-05 2024-09-11T08:44:35.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160905+0000", 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1477000000000005e-05 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.162544+0000", 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016393390000000001 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163580+0000", 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010360300000000001 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163643+0000", 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3034000000000007e-05 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163681+0000", 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7636999999999999e-05 2024-09-11T08:44:35.260 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:25 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4096~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160700+0000", 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0676422849999998, 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031323620000000001, 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 25 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160700+0000", 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160700+0000", 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160700+0000", 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5899999999999996e-07 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160706+0000", 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9100000000000002e-06 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160715+0000", 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2509999999999994e-06 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160721+0000", 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0589999999999999e-06 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.161175+0000", 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00045436100000000002 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.161273+0000", 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7489000000000001e-05 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.161409+0000", 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013657 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163543+0000", 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021335799999999999 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163748+0000", 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00020533699999999999 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163809+0000", 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0727000000000003e-05 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163832+0000", 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3519e-05 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:26 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 8192~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.266 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160732+0000", 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.067610384, 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036173020000000002, 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 26 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160732+0000", 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160732+0000", 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160732+0000", 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0699999999999997e-07 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160749+0000", 2024-09-11T08:44:35.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6813e-05 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160758+0000", 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3790000000000006e-06 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160764+0000", 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.161670+0000", 2024-09-11T08:44:35.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090650400000000003 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.161756+0000", 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5638999999999998e-05 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.161905+0000", 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014886599999999999 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163546+0000", 2024-09-11T08:44:35.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016418400000000001 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164301+0000", 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075468599999999996 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164329+0000", 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7943e-05 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164349+0000", 2024-09-11T08:44:35.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0134999999999999e-05 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:27 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 12288~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160781+0000", 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0675616970000004, 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0048360060000000003, 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.272 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 27 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160781+0000", 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160781+0000", 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160781+0000", 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.82e-07 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160793+0000", 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1459e-05 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160801+0000", 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9680000000000006e-06 2024-09-11T08:44:35.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160807+0000", 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1290000000000001e-06 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.162178+0000", 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001371461 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.162226+0000", 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7794999999999999e-05 2024-09-11T08:44:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.162310+0000", 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3918999999999994e-05 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163841+0000", 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015309639999999999 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165543+0000", 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017026929999999999 2024-09-11T08:44:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165573+0000", 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9748999999999999e-05 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165617+0000", 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3287000000000001e-05 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:28 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 16384~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160816+0000", 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0675259710000002, 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0048944829999999998, 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160816+0000", 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160816+0000", 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160817+0000", 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4499999999999997e-07 2024-09-11T08:44:35.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160824+0000", 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3069999999999997e-06 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160831+0000", 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3370000000000002e-06 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160837+0000", 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7799999999999997e-06 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.162579+0000", 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001742169 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.162671+0000", 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1263999999999999e-05 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.162817+0000", 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014627100000000001 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163843+0000", 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001026292 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165653+0000", 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018101300000000001 2024-09-11T08:44:35.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165684+0000", 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0236999999999999e-05 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165711+0000", 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7151000000000001e-05 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:29 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 20480~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160847+0000", 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0674952519999996, 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049743410000000002, 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 29 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.284 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160847+0000", 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160847+0000", 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160848+0000", 2024-09-11T08:44:35.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5000000000000002e-07 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160854+0000", 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.494e-06 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160861+0000", 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.182e-06 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160867+0000", 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4859999999999997e-06 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163089+0000", 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022219200000000001 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163188+0000", 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9734000000000004e-05 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163327+0000", 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000138398 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165120+0000", 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017931690000000001 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165749+0000", 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00062901000000000005 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165778+0000", 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9329e-05 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165821+0000", 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2969000000000001e-05 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:30 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 24576~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160874+0000", 2024-09-11T08:44:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0674679469999999, 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0050343480000000001, 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 30 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160874+0000", 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160874+0000", 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160875+0000", 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5699999999999998e-07 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160881+0000", 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160888+0000", 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3749999999999997e-06 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160894+0000", 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5879999999999997e-06 2024-09-11T08:44:35.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163869+0000", 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002975409 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.163952+0000", 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2671999999999997e-05 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164044+0000", 2024-09-11T08:44:35.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2128999999999998e-05 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165126+0000", 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001081554 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165854+0000", 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072880299999999998 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165883+0000", 2024-09-11T08:44:35.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8889000000000001e-05 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165909+0000", 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5352999999999999e-05 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:31 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 28672~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160901+0000", 2024-09-11T08:44:35.295 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0674410620000003, 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0055086759999999997, 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 31 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160901+0000", 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160901+0000", 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160902+0000", 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5900000000000002e-07 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160909+0000", 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2150000000000004e-06 2024-09-11T08:44:35.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160916+0000", 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0650000000000001e-06 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160921+0000", 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9899999999999997e-06 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164382+0000", 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034613539999999998 2024-09-11T08:44:35.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164441+0000", 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9206999999999997e-05 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164516+0000", 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4690000000000005e-05 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165130+0000", 2024-09-11T08:44:35.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061353899999999997 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.166361+0000", 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012313129999999999 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.166389+0000", 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8422e-05 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.166410+0000", 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0421999999999999e-05 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:32 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 32768~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160930+0000", 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0674125380000001, 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0055919050000000003, 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.301 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 32 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160930+0000", 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160930+0000", 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160930+0000", 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2899999999999999e-07 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160936+0000", 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160943+0000", 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0049999999999998e-06 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160948+0000", 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6439999999999999e-06 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164747+0000", 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037992019999999998 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164809+0000", 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1926000000000003e-05 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.164890+0000", 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1233e-05 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165524+0000", 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063398999999999997 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.166451+0000", 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092685800000000002 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.166484+0000", 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3461999999999999e-05 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.166522+0000", 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7384e-05 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:33 2.6 2:61e4f98e:::rbd_data.13daa594a2f0.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,write 36864~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.160955+0000", 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0673872810000002, 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062587420000000003, 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 33 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160955+0000", 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160955+0000", 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160955+0000", 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4000000000000002e-07 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160963+0000", 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5279999999999998e-06 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160970+0000", 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1400000000000002e-06 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.160975+0000", 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.288e-06 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165159+0000", 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041838359999999998 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165211+0000", 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1875000000000001e-05 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165290+0000", 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8899000000000005e-05 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.165963+0000", 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00067259100000000005 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.167149+0000", 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011867309999999999 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.167179+0000", 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9405999999999999e-05 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.167214+0000", 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5008000000000001e-05 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.312 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4892.0:53 2.6 2:64b93aac:::rbd_header.13daa594a2f0:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:30.649776+0000", 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.5785657320000004, 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045659840000000004, 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4892", 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/3132814978", 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 53 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.649776+0000", 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.649776+0000", 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.649779+0000", 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.244e-06 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.649799+0000", 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0279999999999999e-05 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.649823+0000", 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3859e-05 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.649836+0000", 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2670000000000001e-05 2024-09-11T08:44:35.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.649920+0000", 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4667999999999996e-05 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.649927+0000", 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8170000000000001e-06 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.652754+0000", 2024-09-11T08:44:35.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00282716 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.652820+0000", 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5788000000000001e-05 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.653055+0000", 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000235496 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-11T08:44:35.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.653515+0000", 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00045921500000000001 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.654283+0000", 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000768288 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.654317+0000", 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3618000000000002e-05 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:30.654342+0000", 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5880999999999998e-05 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:101 2.6 2:7aa5bc23:::rbd_data.14f76f1ec7d3.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.275754+0000", 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.95258790299999996, 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002844769, 2024-09-11T08:44:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 101 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.275754+0000", 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.275754+0000", 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.275755+0000", 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0299999999999999e-07 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.275757+0000", 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6800000000000002e-06 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.275763+0000", 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6679999999999998e-06 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.275767+0000", 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7299999999999999e-06 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.275836+0000", 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9074999999999998e-05 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.275974+0000", 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013824200000000001 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.278599+0000", 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002624871 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.323 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:102 2.6 2:74942152:::rbd_data.14f76f1ec7d3.000000000000001f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.276925+0000", 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.95141708000000003, 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0044049980000000002, 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:35.324 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 102 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.276925+0000", 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.276925+0000", 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.276926+0000", 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4000000000000001e-07 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.276930+0000", 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.143e-06 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.276940+0000", 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0370999999999999e-05 2024-09-11T08:44:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.276946+0000", 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1780000000000002e-06 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.276987+0000", 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1804999999999997e-05 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.277061+0000", 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3473999999999995e-05 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281330+0000", 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042692870000000001 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:107 2.6 2:608c18ea:::rbd_data.14f76f1ec7d3.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:35.328 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.281320+0000", 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.947022154, 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030856479999999999, 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 107 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281320+0000", 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281320+0000", 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281321+0000", 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.052e-06 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281326+0000", 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1420000000000001e-06 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281338+0000", 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.167e-05 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281345+0000", 2024-09-11T08:44:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5660000000000001e-06 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281382+0000", 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6313000000000003e-05 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.281470+0000", 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8404000000000003e-05 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.284406+0000", 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029355010000000001 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:129 2.6 2:6987c2e9:::rbd_data.14f76f1ec7d3.000000000000003a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:35.333 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.292004+0000", 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.93633857300000001, 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039288810000000004, 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 129 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.292004+0000", 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.292004+0000", 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.292004+0000", 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7599999999999996e-07 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.292008+0000", 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.642e-06 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.292021+0000", 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.342e-05 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.292034+0000", 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2482e-05 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.292087+0000", 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3372999999999999e-05 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.292233+0000", 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014613000000000001 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.295933+0000", 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036990579999999999 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.5418.0:131 2.6 2:74204d56:::rbd_data.14f76f1ec7d3.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e61)", 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-11T08:44:34.293123+0000", 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.93521880199999996, 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0056131319999999998, 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-11T08:44:35.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.5418", 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/813409098", 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 131 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.293123+0000", 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.293123+0000", 2024-09-11T08:44:35.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.293124+0000", 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2999999999999999e-07 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.293129+0000", 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9119999999999997e-06 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.293140+0000", 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1085e-05 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.293148+0000", 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3309999999999998e-06 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.293228+0000", 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0488999999999995e-05 2024-09-11T08:44:35.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.293342+0000", 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000114319 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-11T08:44:34.298737+0000", 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0053940660000000003 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-11T08:44:35.342 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-11T08:44:35.343 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:35.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T08:44:35.345 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:35.345+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:35.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:35.485 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:35.485+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:35.493 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:44:35.494 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-11T08:44:35.494 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-11T08:44:35.494 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:44:35.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T08:44:35.545 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:35.545+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:35.567 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-11T08:44:35.567 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:35.585 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-11T08:44:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:35.601 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-11T08:44:35.602 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:35.618 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-11T08:44:35.618 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:35.635 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-11T08:44:35.635 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:35.646 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:35.645+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:35.652 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:35.652 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:35.653 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-11T08:44:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:35.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T08:44:35.668 INFO:tasks.workunit.client.0.smithi045.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-09-11T08:44:35.673 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdclonep47013 2024-09-11T08:44:35.675 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-11T08:44:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:35.694 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-11T08:44:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:35.747 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:35.747+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:35.754 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete...2024-09-11T08:44:35.754+0000 7fd2a4f48640 0 -- 172.21.15.45:0/1414877424 >> [v2:172.21.15.45:6808/2540486849,v1:172.21.15.45:6809/2540486849] conn(0x7fd280008c50 msgr2=0x7fd28000b110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:35.761 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete...2024-09-11T08:44:35.760+0000 7fd2a5749640 0 -- 172.21.15.45:0/1414877424 >> [v2:172.21.15.86:6824/498386186,v1:172.21.15.86:6825/498386186] conn(0x56084f3c9110 msgr2=0x56084f3cb5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:35.771 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-09-11T08:44:35.775 INFO:tasks.workunit.client.0.smithi045.stderr:+ set +x 2024-09-11T08:44:35.775 INFO:tasks.workunit.client.0.smithi045.stderr:+ test_rbd_import 2024-09-11T08:44:35.775 INFO:tasks.workunit.client.0.smithi045.stderr:+ local src=testrbdimports47013 2024-09-11T08:44:35.775 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd create --size 256 testrbdimports47013 2024-09-11T08:44:35.792 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:35.793 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:35.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T08:44:35.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd export testrbdimports47013 /tmp/rbd_journal47013/testrbdimports47013.export 2024-09-11T08:44:35.863 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:35.863+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:35.914 INFO:tasks.workunit.client.0.smithi045.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2024-09-11T08:44:35.919 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdimports47013 2024-09-11T08:44:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:35.949 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:35.948+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:35.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T08:44:36.003 INFO:tasks.workunit.client.0.smithi045.stderr:2024-09-11T08:44:36.002+0000 7f84a9274640 0 -- 172.21.15.45:0/1208993413 >> [v2:172.21.15.45:6808/2540486849,v1:172.21.15.45:6809/2540486849] conn(0x7f8494071730 msgr2=0x7f8494073b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:36.015 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-09-11T08:44:36.019 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdimport47013 2024-09-11T08:44:36.019 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd import --image-feature exclusive-lock --image-feature journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 /tmp/rbd_journal47013/testrbdimports47013.export testrbdimport47013 2024-09-11T08:44:36.049 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:36.050+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:36.106 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:36.106 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:36.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T08:44:36.150 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-09-11T08:44:36.151+0000 7fc6b1d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33419) UID: 0 2024-09-11T08:44:36.221 INFO:tasks.workunit.client.0.smithi045.stderr: Importing image: 1% complete... Importing image: 3% complete... Importing image: 4% complete... Importing image: 6% complete... Importing image: 7% complete... Importing image: 9% complete... Importing image: 10% complete... Importing image: 12% complete... Importing image: 14% complete... Importing image: 15% complete... Importing image: 17% complete... Importing image: 18% complete... Importing image: 20% complete... Importing image: 21% complete... Importing image: 23% complete... Importing image: 25% complete... Importing image: 26% complete... Importing image: 28% complete... Importing image: 29% complete... Importing image: 31% complete... Importing image: 32% complete... Importing image: 34% complete... Importing image: 35% complete... Importing image: 37% complete... Importing image: 39% complete... Importing image: 40% complete... Importing image: 42% complete... Importing image: 43% complete... Importing image: 45% complete... Importing image: 46% complete... Importing image: 48% complete... Importing image: 50% complete... Importing image: 51% complete... Importing image: 53% complete... Importing image: 54% complete... Importing image: 56% complete... Importing image: 57% complete... Importing image: 59% complete... Importing image: 60% complete... Importing image: 62% complete... Importing image: 64% complete... Importing image: 65% complete... Importing image: 67% complete... Importing image: 68% complete... Importing image: 70% complete... Importing image: 71% complete... Importing image: 73% complete... Importing image: 75% complete... Importing image: 76% complete... Importing image: 78% complete... Importing image: 79% complete... Importing image: 81% complete... Importing image: 82% complete... Importing image: 84% complete... Importing image: 85% complete... Importing image: 87% complete... Importing image: 89% complete... Importing image: 90% complete... Importing image: 92% complete... Importing image: 93% complete... Importing image: 95% complete... Importing image: 96% complete... Importing image: 98% complete... Importing image: 100% complete...done. 2024-09-11T08:44:36.225 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdimport47013 'journal info' //journal/order 25 2024-09-11T08:44:36.225 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdimport47013 2024-09-11T08:44:36.225 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:36.225 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/order 2024-09-11T08:44:36.225 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=25 2024-09-11T08:44:36.226 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdimport47013 2024-09-11T08:44:36.226 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/order 2024-09-11T08:44:36.240 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:36.240 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:36.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T08:44:36.280 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=25 2024-09-11T08:44:36.280 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 25 = 25 2024-09-11T08:44:36.280 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdimport47013 'journal info' //journal/splay_width 6 2024-09-11T08:44:36.280 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdimport47013 2024-09-11T08:44:36.280 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:36.281 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/splay_width 2024-09-11T08:44:36.281 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=6 2024-09-11T08:44:36.281 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:36.279+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:36.281 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdimport47013 2024-09-11T08:44:36.281 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-09-11T08:44:36.334 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=6 2024-09-11T08:44:36.334 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 6 = 6 2024-09-11T08:44:36.334 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdimport47013 'journal info' //journal/object_pool rbd 2024-09-11T08:44:36.334 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdimport47013 2024-09-11T08:44:36.334 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:36.334 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/object_pool 2024-09-11T08:44:36.334 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=rbd 2024-09-11T08:44:36.335 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdimport47013 2024-09-11T08:44:36.335 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-09-11T08:44:36.351 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:36.351+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:36.357 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:36.358 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:36.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T08:44:36.387 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=rbd 2024-09-11T08:44:36.387 INFO:tasks.workunit.client.0.smithi045.stderr:+ test rbd = rbd 2024-09-11T08:44:36.387 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdimport47013 2024-09-11T08:44:36.453 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-09-11T08:44:36.453+0000 7f2eecc40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33414) UID: 0 2024-09-11T08:44:36.461 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete...2024-09-11T08:44:36.460+0000 7f50f175b640 0 -- 172.21.15.45:0/3324920001 >> [v2:172.21.15.45:6824/1040844445,v1:172.21.15.45:6825/1040844445] conn(0x55b6d6a65900 msgr2=0x55b6d6a67cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:36.464 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete...2024-09-11T08:44:36.463+0000 7f50efcf8640 0 -- 172.21.15.45:0/3324920001 >> [v2:172.21.15.86:6816/2975777507,v1:172.21.15.86:6817/2975777507] conn(0x7f50cc002ce0 msgr2=0x7f50cc00ce70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:36.478 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:36.478 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:36.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T08:44:36.492 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-09-11T08:44:36.496 INFO:tasks.workunit.client.0.smithi045.stderr:+ set +x 2024-09-11T08:44:36.496 INFO:tasks.workunit.client.0.smithi045.stderr:+ test_rbd_feature 2024-09-11T08:44:36.496 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdfeature47013 2024-09-11T08:44:36.496 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd create --image-feature exclusive-lock --size 256 testrbdfeature47013 2024-09-11T08:44:36.545 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd feature enable testrbdfeature47013 journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 2024-09-11T08:44:36.553 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:36.553+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:36.612 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdfeature47013 'journal info' //journal/order 25 2024-09-11T08:44:36.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdfeature47013 2024-09-11T08:44:36.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:36.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/order 2024-09-11T08:44:36.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=25 2024-09-11T08:44:36.613 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdfeature47013 2024-09-11T08:44:36.613 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/order 2024-09-11T08:44:36.633 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:36.633 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:36.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T08:44:36.654 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:36.654+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:36.667 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=25 2024-09-11T08:44:36.668 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 25 = 25 2024-09-11T08:44:36.668 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdfeature47013 'journal info' //journal/splay_width 6 2024-09-11T08:44:36.668 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdfeature47013 2024-09-11T08:44:36.668 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:36.668 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/splay_width 2024-09-11T08:44:36.668 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=6 2024-09-11T08:44:36.668 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdfeature47013 2024-09-11T08:44:36.668 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-09-11T08:44:36.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T08:44:36.722 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=6 2024-09-11T08:44:36.722 INFO:tasks.workunit.client.0.smithi045.stderr:+ test 6 = 6 2024-09-11T08:44:36.722 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd_assert_eq testrbdfeature47013 'journal info' //journal/object_pool rbd 2024-09-11T08:44:36.722 INFO:tasks.workunit.client.0.smithi045.stderr:+ local image=testrbdfeature47013 2024-09-11T08:44:36.722 INFO:tasks.workunit.client.0.smithi045.stderr:+ local 'cmd=journal info' 2024-09-11T08:44:36.722 INFO:tasks.workunit.client.0.smithi045.stderr:+ local param=//journal/object_pool 2024-09-11T08:44:36.722 INFO:tasks.workunit.client.0.smithi045.stderr:+ local expected_val=rbd 2024-09-11T08:44:36.723 INFO:tasks.workunit.client.0.smithi045.stderr:++ rbd --format xml journal info --image testrbdfeature47013 2024-09-11T08:44:36.723 INFO:tasks.workunit.client.0.smithi045.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-09-11T08:44:36.748 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:36.748 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:36.755 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:36.755+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:36.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T08:44:36.777 INFO:tasks.workunit.client.0.smithi045.stderr:+ local val=rbd 2024-09-11T08:44:36.777 INFO:tasks.workunit.client.0.smithi045.stderr:+ test rbd = rbd 2024-09-11T08:44:36.777 INFO:tasks.workunit.client.0.smithi045.stderr:+ rbd remove testrbdfeature47013 2024-09-11T08:44:36.856 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-11T08:44:36.855+0000 7efd01f44640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33226) UID: 0 2024-09-11T08:44:36.863 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:36.863 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:36.866 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image2024-09-11T08:44:36.864+0000 7f2813cb7640 0 -- 172.21.15.45:0/3212506898 >> [v2:172.21.15.45:6824/1040844445,v1:172.21.15.45:6825/1040844445] conn(0x5578877706a0 msgr2=0x557887772a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:36.871 INFO:tasks.workunit.client.0.smithi045.stderr:: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete...2024-09-11T08:44:36.870+0000 7f28144b8640 0 -- 172.21.15.45:0/3212506898 >> [v2:172.21.15.86:6824/498386186,v1:172.21.15.86:6825/498386186] conn(0x5578877c30a0 msgr2=0x5578877c5530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T08:44:36.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T08:44:36.909 INFO:tasks.workunit.client.0.smithi045.stderr: Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-09-11T08:44:36.913 INFO:tasks.workunit.client.0.smithi045.stderr:+ set +x 2024-09-11T08:44:36.913 INFO:tasks.workunit.client.0.smithi045.stdout:OK 2024-09-11T08:44:36.946 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T08:44:36.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T08:44:36.957 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-09-11T08:44:36.957+0000 7f2e8275f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33418) UID: 0 2024-09-11T08:44:36.980 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:36.980 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:36.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-11T08:44:37.014 INFO:tasks.workunit:Stopping ['rbd/journal.sh'] on client.0... 2024-09-11T08:44:37.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T08:44:37.057 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:37.056+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:37.134 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:37.134 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:37.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-11T08:44:37.158 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-11T08:44:37.158+0000 7ff59a30a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33220) UID: 0 2024-09-11T08:44:37.246 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:37.246 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:37.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-11T08:44:37.298 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-09-11T08:44:37.299+0000 7f43a8ca2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33423) UID: 0 2024-09-11T08:44:37.334 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:37.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:44:37.358 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:37.358 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:37.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-11T08:44:37.375 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-11T08:44:37.375+0000 7f8ddf7f5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33225) UID: 0 2024-09-11T08:44:37.400 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:44:37.400 DEBUG:teuthology.orchestra.run.smithi045:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:44:37.465 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:44:37.465 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-11T08:44:37.474 INFO:tasks.thrashosds:joining thrashosds 2024-09-11T08:44:37.475 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-11T08:44:37.464+0000 7fee2bfdd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33228) UID: 0 2024-09-11T08:44:37.511 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:37.511 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:37.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-11T08:44:37.623 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:37.623 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:37.631 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-11T08:44:37.734 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:37.734 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:37.988 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-11T08:44:38.037 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-11T08:44:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a: {} 2024-09-11T08:44:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:44:38.172 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b: {} 2024-09-11T08:44:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:44:38.181 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c: {} 2024-09-11T08:44:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:44:38.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T08:44:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T08:44:39.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-11T08:44:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-11T08:44:40.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T08:44:40.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-11T08:44:40.996 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-11T08:44:40.997 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T08:44:41.017 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T08:44:41.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-11T08:44:42.003 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-11T08:44:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-11T08:44:42.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T08:44:43.002 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T08:44:43.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-11T08:44:44.008 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-11T08:44:44.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T08:44:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T08:44:45.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-11T08:44:46.019 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-11T08:44:46.037 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T08:44:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T08:44:46.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-11T08:44:48.031 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-11T08:44:48.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T08:44:49.036 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T08:44:49.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-11T08:44:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-11T08:44:50.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-11T08:44:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.6 to 1 (10000) 2024-09-11T08:44:51.063 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-11T08:44:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-11T08:44:52.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-11T08:44:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.7 to 1 (10000) 2024-09-11T08:44:53.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-11T08:44:54.066 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-11T08:44:54.084 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-11T08:44:54.084 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-11T08:44:54.084 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-11T08:44:54.084 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-11T08:44:54.084 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T08:44:54.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T08:44:54.384 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:54.384 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:44:54.063026+0000","last_up_change":"2024-09-11T08:43:09.343672+0000","last_in_change":"2024-09-11T08:44:27.636062+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:43:08.201248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T08:43:16.189979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":0.75,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6809","nonce":2540486849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6811","nonce":2540486849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6815","nonce":2540486849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6813","nonce":2540486849}]},"public_addr":"172.21.15.45:6809/2540486849","cluster_addr":"172.21.15.45:6811/2540486849","heartbeat_back_addr":"172.21.15.45:6815/2540486849","heartbeat_front_addr":"172.21.15.45:6813/2540486849","state":["exists","up"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6825","nonce":1040844445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6827","nonce":1040844445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6831","nonce":1040844445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6829","nonce":1040844445}]},"public_addr":"172.21.15.45:6825/1040844445","cluster_addr":"172.21.15.45:6827/1040844445","heartbeat_back_addr":"172.21.15.45:6831/1040844445","heartbeat_front_addr":"172.21.15.45:6829/1040844445","state":["exists","up"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6817","nonce":1231172466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6819","nonce":1231172466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6823","nonce":1231172466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6821","nonce":1231172466}]},"public_addr":"172.21.15.45:6817/1231172466","cluster_addr":"172.21.15.45:6819/1231172466","heartbeat_back_addr":"172.21.15.45:6823/1231172466","heartbeat_front_addr":"172.21.15.45:6821/1231172466","state":["exists","up"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6801","nonce":226138367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6803","nonce":226138367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6807","nonce":226138367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6805","nonce":226138367}]},"public_addr":"172.21.15.45:6801/226138367","cluster_addr":"172.21.15.45:6803/226138367","heartbeat_back_addr":"172.21.15.45:6807/226138367","heartbeat_front_addr":"172.21.15.45:6805/226138367","state":["exists","up"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6801","nonce":374760845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6803","nonce":374760845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6807","nonce":374760845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6805","nonce":374760845}]},"public_addr":"172.21.15.86:6801/374760845","cluster_addr":"172.21.15.86:6803/374760845","heartbeat_back_addr":"172.21.15.86:6807/374760845","heartbeat_front_addr":"172.21.15.86:6805/374760845","state":["exists","up"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6825","nonce":498386186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6827","nonce":498386186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6831","nonce":498386186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6829","nonce":498386186}]},"public_addr":"172.21.15.86:6825/498386186","cluster_addr":"172.21.15.86:6827/498386186","heartbeat_back_addr":"172.21.15.86:6831/498386186","heartbeat_front_addr":"172.21.15.86:6829/498386186","state":["exists","up"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6809","nonce":1261869816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6811","nonce":1261869816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6815","nonce":1261869816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6813","nonce":1261869816}]},"public_addr":"172.21.15.86:6809/1261869816","cluster_addr":"172.21.15.86:6811/1261869816","heartbeat_back_addr":"172.21.15.86:6815/1261869816","heartbeat_front_addr":"172.21.15.86:6813/1261869816","state":["exists","up"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6817","nonce":2975777507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6819","nonce":2975777507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6823","nonce":2975777507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6821","nonce":2975777507}]},"public_addr":"172.21.15.86:6817/2975777507","cluster_addr":"172.21.15.86:6819/2975777507","heartbeat_back_addr":"172.21.15.86:6823/2975777507","heartbeat_front_addr":"172.21.15.86:6821/2975777507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.394370+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.813228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.425286+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.147591+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:07.119248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.653838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.459308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.492589+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.0","osds":[2,0]},{"pgid":"2.1","osds":[0,1]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:44:54.398 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T08:44:54.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T08:44:54.692 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:54.692 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:44:54.063026+0000","last_up_change":"2024-09-11T08:43:09.343672+0000","last_in_change":"2024-09-11T08:44:27.636062+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:43:08.201248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T08:43:16.189979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":0.75,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6809","nonce":2540486849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6811","nonce":2540486849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6815","nonce":2540486849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6813","nonce":2540486849}]},"public_addr":"172.21.15.45:6809/2540486849","cluster_addr":"172.21.15.45:6811/2540486849","heartbeat_back_addr":"172.21.15.45:6815/2540486849","heartbeat_front_addr":"172.21.15.45:6813/2540486849","state":["exists","up"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6825","nonce":1040844445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6827","nonce":1040844445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6831","nonce":1040844445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6829","nonce":1040844445}]},"public_addr":"172.21.15.45:6825/1040844445","cluster_addr":"172.21.15.45:6827/1040844445","heartbeat_back_addr":"172.21.15.45:6831/1040844445","heartbeat_front_addr":"172.21.15.45:6829/1040844445","state":["exists","up"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6817","nonce":1231172466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6819","nonce":1231172466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6823","nonce":1231172466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6821","nonce":1231172466}]},"public_addr":"172.21.15.45:6817/1231172466","cluster_addr":"172.21.15.45:6819/1231172466","heartbeat_back_addr":"172.21.15.45:6823/1231172466","heartbeat_front_addr":"172.21.15.45:6821/1231172466","state":["exists","up"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6801","nonce":226138367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6803","nonce":226138367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6807","nonce":226138367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6805","nonce":226138367}]},"public_addr":"172.21.15.45:6801/226138367","cluster_addr":"172.21.15.45:6803/226138367","heartbeat_back_addr":"172.21.15.45:6807/226138367","heartbeat_front_addr":"172.21.15.45:6805/226138367","state":["exists","up"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6801","nonce":374760845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6803","nonce":374760845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6807","nonce":374760845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6805","nonce":374760845}]},"public_addr":"172.21.15.86:6801/374760845","cluster_addr":"172.21.15.86:6803/374760845","heartbeat_back_addr":"172.21.15.86:6807/374760845","heartbeat_front_addr":"172.21.15.86:6805/374760845","state":["exists","up"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6825","nonce":498386186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6827","nonce":498386186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6831","nonce":498386186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6829","nonce":498386186}]},"public_addr":"172.21.15.86:6825/498386186","cluster_addr":"172.21.15.86:6827/498386186","heartbeat_back_addr":"172.21.15.86:6831/498386186","heartbeat_front_addr":"172.21.15.86:6829/498386186","state":["exists","up"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6809","nonce":1261869816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6811","nonce":1261869816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6815","nonce":1261869816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6813","nonce":1261869816}]},"public_addr":"172.21.15.86:6809/1261869816","cluster_addr":"172.21.15.86:6811/1261869816","heartbeat_back_addr":"172.21.15.86:6815/1261869816","heartbeat_front_addr":"172.21.15.86:6813/1261869816","state":["exists","up"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6817","nonce":2975777507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6819","nonce":2975777507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6823","nonce":2975777507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6821","nonce":2975777507}]},"public_addr":"172.21.15.86:6817/2975777507","cluster_addr":"172.21.15.86:6819/2975777507","heartbeat_back_addr":"172.21.15.86:6823/2975777507","heartbeat_front_addr":"172.21.15.86:6821/2975777507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.394370+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.813228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.425286+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.147591+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:07.119248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.653838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.459308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.492589+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.0","osds":[2,0]},{"pgid":"2.1","osds":[0,1]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:44:54.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T08:44:54.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T08:44:54.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T08:44:54.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-11T08:44:54.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-11T08:44:54.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-11T08:44:54.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-11T08:44:54.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-11T08:44:54.958 INFO:teuthology.orchestra.run.smithi045.stdout:55834574872 2024-09-11T08:44:54.958 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T08:44:54.962 INFO:teuthology.orchestra.run.smithi045.stdout:64424509463 2024-09-11T08:44:54.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T08:44:54.963 INFO:teuthology.orchestra.run.smithi045.stdout:55834574873 2024-09-11T08:44:54.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T08:44:54.965 INFO:teuthology.orchestra.run.smithi045.stdout:64424509464 2024-09-11T08:44:54.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-11T08:44:54.967 INFO:teuthology.orchestra.run.smithi045.stdout:64424509463 2024-09-11T08:44:54.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T08:44:54.968 INFO:teuthology.orchestra.run.smithi045.stdout:64424509463 2024-09-11T08:44:54.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-11T08:44:54.969 INFO:teuthology.orchestra.run.smithi045.stdout:64424509465 2024-09-11T08:44:54.969 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T08:44:54.970 INFO:teuthology.orchestra.run.smithi045.stdout:64424509465 2024-09-11T08:44:54.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T08:44:55.539 INFO:teuthology.orchestra.run.smithi045.stdout:55834574871 2024-09-11T08:44:55.546 INFO:teuthology.orchestra.run.smithi045.stdout:64424509462 2024-09-11T08:44:55.558 INFO:teuthology.orchestra.run.smithi045.stdout:64424509463 2024-09-11T08:44:55.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574872 got 55834574871 for osd.4 2024-09-11T08:44:55.570 INFO:teuthology.orchestra.run.smithi045.stdout:55834574872 2024-09-11T08:44:55.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509462 for osd.5 2024-09-11T08:44:55.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509464 got 64424509463 for osd.6 2024-09-11T08:44:55.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574873 got 55834574872 for osd.3 2024-09-11T08:44:55.591 INFO:teuthology.orchestra.run.smithi045.stdout:64424509462 2024-09-11T08:44:55.595 INFO:teuthology.orchestra.run.smithi045.stdout:64424509462 2024-09-11T08:44:55.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509462 for osd.0 2024-09-11T08:44:55.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509462 for osd.7 2024-09-11T08:44:55.627 INFO:teuthology.orchestra.run.smithi045.stdout:64424509464 2024-09-11T08:44:55.637 INFO:teuthology.orchestra.run.smithi045.stdout:64424509464 2024-09-11T08:44:55.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509465 got 64424509464 for osd.2 2024-09-11T08:44:55.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509465 got 64424509464 for osd.1 2024-09-11T08:44:56.562 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T08:44:56.571 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T08:44:56.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-11T08:44:56.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T08:44:56.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T08:44:56.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-11T08:44:56.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T08:44:56.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T08:44:57.123 INFO:teuthology.orchestra.run.smithi045.stdout:55834574872 2024-09-11T08:44:57.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574872 got 55834574872 for osd.4 2024-09-11T08:44:57.144 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:57.189 INFO:teuthology.orchestra.run.smithi045.stdout:64424509466 2024-09-11T08:44:57.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509465 got 64424509466 for osd.2 2024-09-11T08:44:57.204 INFO:teuthology.orchestra.run.smithi045.stdout:55834574873 2024-09-11T08:44:57.204 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:57.216 INFO:teuthology.orchestra.run.smithi045.stdout:64424509464 2024-09-11T08:44:57.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574873 got 55834574873 for osd.3 2024-09-11T08:44:57.222 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:57.228 INFO:teuthology.orchestra.run.smithi045.stdout:64424509465 2024-09-11T08:44:57.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509464 for osd.5 2024-09-11T08:44:57.235 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:57.241 INFO:teuthology.orchestra.run.smithi045.stdout:64424509466 2024-09-11T08:44:57.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509464 got 64424509465 for osd.6 2024-09-11T08:44:57.246 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:57.250 INFO:teuthology.orchestra.run.smithi045.stdout:64424509464 2024-09-11T08:44:57.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509465 got 64424509466 for osd.1 2024-09-11T08:44:57.255 INFO:teuthology.orchestra.run.smithi045.stdout:64424509464 2024-09-11T08:44:57.255 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:57.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509464 for osd.0 2024-09-11T08:44:57.265 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:57.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509464 for osd.7 2024-09-11T08:44:57.269 DEBUG:teuthology.parallel:result is None 2024-09-11T08:44:57.269 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T08:44:57.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:44:57.596 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:44:57.608 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-11T08:44:56.168005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2962,"num_read_kb":2803,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1093,"ondisk_log_size":1093,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":266400,"kb_used_data":8548,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708320,"statfs":{"total":773094113280,"available":772821319680,"internally_reserved":0,"allocated":8753152,"data_stored":4393338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.121061"},"pg_stats":[{"pgid":"2.7","version":"63'93","reported_seq":245,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336654+0000","last_change":"2024-09-11T08:44:35.753591+0000","last_active":"2024-09-11T08:44:54.336654+0000","last_peered":"2024-09-11T08:44:54.336654+0000","last_clean":"2024-09-11T08:44:54.336654+0000","last_became_active":"2024-09-11T08:44:28.934797+0000","last_became_peered":"2024-09-11T08:44:28.934797+0000","last_unstale":"2024-09-11T08:44:54.336654+0000","last_undegraded":"2024-09-11T08:44:54.336654+0000","last_fullsized":"2024-09-11T08:44:54.336654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:50.968529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027236169999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":154,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"63'135","reported_seq":411,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335927+0000","last_change":"2024-09-11T08:44:35.662727+0000","last_active":"2024-09-11T08:44:54.335927+0000","last_peered":"2024-09-11T08:44:54.335927+0000","last_clean":"2024-09-11T08:44:54.335927+0000","last_became_active":"2024-09-11T08:44:28.650336+0000","last_became_peered":"2024-09-11T08:44:28.650336+0000","last_unstale":"2024-09-11T08:44:54.335927+0000","last_undegraded":"2024-09-11T08:44:54.335927+0000","last_fullsized":"2024-09-11T08:44:54.335927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:47.097888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000391343,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":336,"num_write":69,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'73","reported_seq":251,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336525+0000","last_change":"2024-09-11T08:44:35.751095+0000","last_active":"2024-09-11T08:44:54.336525+0000","last_peered":"2024-09-11T08:44:54.336525+0000","last_clean":"2024-09-11T08:44:54.336525+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:44:54.336525+0000","last_undegraded":"2024-09-11T08:44:54.336525+0000","last_fullsized":"2024-09-11T08:44:54.336525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:09.418627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000448348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":154,"num_write":35,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"63'162","reported_seq":483,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335921+0000","last_change":"2024-09-11T08:44:35.662613+0000","last_active":"2024-09-11T08:44:54.335921+0000","last_peered":"2024-09-11T08:44:54.335921+0000","last_clean":"2024-09-11T08:44:54.335921+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:44:54.335921+0000","last_undegraded":"2024-09-11T08:44:54.335921+0000","last_fullsized":"2024-09-11T08:44:54.335921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037972899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":427,"num_read_kb":421,"num_write":107,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"63'98","reported_seq":460,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336448+0000","last_change":"2024-09-11T08:44:35.662980+0000","last_active":"2024-09-11T08:44:54.336448+0000","last_peered":"2024-09-11T08:44:54.336448+0000","last_clean":"2024-09-11T08:44:54.336448+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:44:54.336448+0000","last_undegraded":"2024-09-11T08:44:54.336448+0000","last_fullsized":"2024-09-11T08:44:54.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:20.735037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037767200000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":290,"num_read_kb":266,"num_write":22,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"63'161","reported_seq":508,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.162746+0000","last_change":"2024-09-11T08:44:35.668561+0000","last_active":"2024-09-11T08:44:54.162746+0000","last_peered":"2024-09-11T08:44:54.162746+0000","last_clean":"2024-09-11T08:44:54.162746+0000","last_became_active":"2024-09-11T08:44:11.733798+0000","last_became_peered":"2024-09-11T08:44:11.733798+0000","last_unstale":"2024-09-11T08:44:54.162746+0000","last_undegraded":"2024-09-11T08:44:54.162746+0000","last_fullsized":"2024-09-11T08:44:54.162746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:35:00.417002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096114499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":657,"num_read_kb":590,"num_write":79,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'90","reported_seq":380,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336492+0000","last_change":"2024-09-11T08:44:35.751698+0000","last_active":"2024-09-11T08:44:54.336492+0000","last_peered":"2024-09-11T08:44:54.336492+0000","last_clean":"2024-09-11T08:44:54.336492+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:44:54.336492+0000","last_undegraded":"2024-09-11T08:44:54.336492+0000","last_fullsized":"2024-09-11T08:44:54.336492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:28.421835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010508239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":306,"num_read_kb":258,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"63'164","reported_seq":607,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336462+0000","last_change":"2024-09-11T08:44:35.663067+0000","last_active":"2024-09-11T08:44:54.336462+0000","last_peered":"2024-09-11T08:44:54.336462+0000","last_clean":"2024-09-11T08:44:54.336462+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:44:54.336462+0000","last_undegraded":"2024-09-11T08:44:54.336462+0000","last_fullsized":"2024-09-11T08:44:54.336462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:27.366610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044274400000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":524,"num_write":70,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.066692+0000","last_change":"2024-09-11T08:44:02.988511+0000","last_active":"2024-09-11T08:44:54.066692+0000","last_peered":"2024-09-11T08:44:54.066692+0000","last_clean":"2024-09-11T08:44:54.066692+0000","last_became_active":"2024-09-11T08:44:02.988115+0000","last_became_peered":"2024-09-11T08:44:02.988115+0000","last_unstale":"2024-09-11T08:44:54.066692+0000","last_undegraded":"2024-09-11T08:44:54.066692+0000","last_fullsized":"2024-09-11T08:44:54.066692+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:51.312057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2846,"num_read_kb":2703,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":976,"ondisk_log_size":976,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":933888,"data_stored":396751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:44:57.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:44:57.898 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:57.898 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:44:57.910 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-11T08:44:56.168005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2962,"num_read_kb":2803,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1093,"ondisk_log_size":1093,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":266400,"kb_used_data":8548,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708320,"statfs":{"total":773094113280,"available":772821319680,"internally_reserved":0,"allocated":8753152,"data_stored":4393338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.121061"},"pg_stats":[{"pgid":"2.7","version":"63'93","reported_seq":245,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336654+0000","last_change":"2024-09-11T08:44:35.753591+0000","last_active":"2024-09-11T08:44:54.336654+0000","last_peered":"2024-09-11T08:44:54.336654+0000","last_clean":"2024-09-11T08:44:54.336654+0000","last_became_active":"2024-09-11T08:44:28.934797+0000","last_became_peered":"2024-09-11T08:44:28.934797+0000","last_unstale":"2024-09-11T08:44:54.336654+0000","last_undegraded":"2024-09-11T08:44:54.336654+0000","last_fullsized":"2024-09-11T08:44:54.336654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:50.968529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027236169999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":154,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"63'135","reported_seq":411,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335927+0000","last_change":"2024-09-11T08:44:35.662727+0000","last_active":"2024-09-11T08:44:54.335927+0000","last_peered":"2024-09-11T08:44:54.335927+0000","last_clean":"2024-09-11T08:44:54.335927+0000","last_became_active":"2024-09-11T08:44:28.650336+0000","last_became_peered":"2024-09-11T08:44:28.650336+0000","last_unstale":"2024-09-11T08:44:54.335927+0000","last_undegraded":"2024-09-11T08:44:54.335927+0000","last_fullsized":"2024-09-11T08:44:54.335927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:47.097888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000391343,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":336,"num_write":69,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'73","reported_seq":251,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336525+0000","last_change":"2024-09-11T08:44:35.751095+0000","last_active":"2024-09-11T08:44:54.336525+0000","last_peered":"2024-09-11T08:44:54.336525+0000","last_clean":"2024-09-11T08:44:54.336525+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:44:54.336525+0000","last_undegraded":"2024-09-11T08:44:54.336525+0000","last_fullsized":"2024-09-11T08:44:54.336525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:09.418627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000448348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":154,"num_write":35,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"63'162","reported_seq":483,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335921+0000","last_change":"2024-09-11T08:44:35.662613+0000","last_active":"2024-09-11T08:44:54.335921+0000","last_peered":"2024-09-11T08:44:54.335921+0000","last_clean":"2024-09-11T08:44:54.335921+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:44:54.335921+0000","last_undegraded":"2024-09-11T08:44:54.335921+0000","last_fullsized":"2024-09-11T08:44:54.335921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037972899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":427,"num_read_kb":421,"num_write":107,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"63'98","reported_seq":460,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336448+0000","last_change":"2024-09-11T08:44:35.662980+0000","last_active":"2024-09-11T08:44:54.336448+0000","last_peered":"2024-09-11T08:44:54.336448+0000","last_clean":"2024-09-11T08:44:54.336448+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:44:54.336448+0000","last_undegraded":"2024-09-11T08:44:54.336448+0000","last_fullsized":"2024-09-11T08:44:54.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:20.735037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037767200000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":290,"num_read_kb":266,"num_write":22,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"63'161","reported_seq":508,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.162746+0000","last_change":"2024-09-11T08:44:35.668561+0000","last_active":"2024-09-11T08:44:54.162746+0000","last_peered":"2024-09-11T08:44:54.162746+0000","last_clean":"2024-09-11T08:44:54.162746+0000","last_became_active":"2024-09-11T08:44:11.733798+0000","last_became_peered":"2024-09-11T08:44:11.733798+0000","last_unstale":"2024-09-11T08:44:54.162746+0000","last_undegraded":"2024-09-11T08:44:54.162746+0000","last_fullsized":"2024-09-11T08:44:54.162746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:35:00.417002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096114499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":657,"num_read_kb":590,"num_write":79,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'90","reported_seq":380,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336492+0000","last_change":"2024-09-11T08:44:35.751698+0000","last_active":"2024-09-11T08:44:54.336492+0000","last_peered":"2024-09-11T08:44:54.336492+0000","last_clean":"2024-09-11T08:44:54.336492+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:44:54.336492+0000","last_undegraded":"2024-09-11T08:44:54.336492+0000","last_fullsized":"2024-09-11T08:44:54.336492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:28.421835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010508239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":306,"num_read_kb":258,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"63'164","reported_seq":607,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336462+0000","last_change":"2024-09-11T08:44:35.663067+0000","last_active":"2024-09-11T08:44:54.336462+0000","last_peered":"2024-09-11T08:44:54.336462+0000","last_clean":"2024-09-11T08:44:54.336462+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:44:54.336462+0000","last_undegraded":"2024-09-11T08:44:54.336462+0000","last_fullsized":"2024-09-11T08:44:54.336462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:27.366610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044274400000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":524,"num_write":70,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.066692+0000","last_change":"2024-09-11T08:44:02.988511+0000","last_active":"2024-09-11T08:44:54.066692+0000","last_peered":"2024-09-11T08:44:54.066692+0000","last_clean":"2024-09-11T08:44:54.066692+0000","last_became_active":"2024-09-11T08:44:02.988115+0000","last_became_peered":"2024-09-11T08:44:02.988115+0000","last_unstale":"2024-09-11T08:44:54.066692+0000","last_undegraded":"2024-09-11T08:44:54.066692+0000","last_fullsized":"2024-09-11T08:44:54.066692+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:51.312057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2846,"num_read_kb":2703,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":976,"ondisk_log_size":976,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":933888,"data_stored":396751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:44:57.911 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T08:44:57.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-11T08:44:58.223 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-11T08:44:58.232 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T08:44:58.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:44:58.521 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:58.521 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:44:58.534 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-11T08:44:58.168443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2962,"num_read_kb":2803,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1093,"ondisk_log_size":1093,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":266416,"kb_used_data":8560,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708304,"statfs":{"total":773094113280,"available":772821303296,"internally_reserved":0,"allocated":8765440,"data_stored":4398846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.113290"},"pg_stats":[{"pgid":"2.7","version":"63'93","reported_seq":245,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336654+0000","last_change":"2024-09-11T08:44:35.753591+0000","last_active":"2024-09-11T08:44:54.336654+0000","last_peered":"2024-09-11T08:44:54.336654+0000","last_clean":"2024-09-11T08:44:54.336654+0000","last_became_active":"2024-09-11T08:44:28.934797+0000","last_became_peered":"2024-09-11T08:44:28.934797+0000","last_unstale":"2024-09-11T08:44:54.336654+0000","last_undegraded":"2024-09-11T08:44:54.336654+0000","last_fullsized":"2024-09-11T08:44:54.336654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:50.968529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027236169999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":154,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"63'135","reported_seq":411,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335927+0000","last_change":"2024-09-11T08:44:35.662727+0000","last_active":"2024-09-11T08:44:54.335927+0000","last_peered":"2024-09-11T08:44:54.335927+0000","last_clean":"2024-09-11T08:44:54.335927+0000","last_became_active":"2024-09-11T08:44:28.650336+0000","last_became_peered":"2024-09-11T08:44:28.650336+0000","last_unstale":"2024-09-11T08:44:54.335927+0000","last_undegraded":"2024-09-11T08:44:54.335927+0000","last_fullsized":"2024-09-11T08:44:54.335927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:47.097888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000391343,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":336,"num_write":69,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'73","reported_seq":251,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336525+0000","last_change":"2024-09-11T08:44:35.751095+0000","last_active":"2024-09-11T08:44:54.336525+0000","last_peered":"2024-09-11T08:44:54.336525+0000","last_clean":"2024-09-11T08:44:54.336525+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:44:54.336525+0000","last_undegraded":"2024-09-11T08:44:54.336525+0000","last_fullsized":"2024-09-11T08:44:54.336525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:09.418627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000448348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":154,"num_write":35,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"63'162","reported_seq":483,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335921+0000","last_change":"2024-09-11T08:44:35.662613+0000","last_active":"2024-09-11T08:44:54.335921+0000","last_peered":"2024-09-11T08:44:54.335921+0000","last_clean":"2024-09-11T08:44:54.335921+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:44:54.335921+0000","last_undegraded":"2024-09-11T08:44:54.335921+0000","last_fullsized":"2024-09-11T08:44:54.335921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037972899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":427,"num_read_kb":421,"num_write":107,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"63'98","reported_seq":460,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336448+0000","last_change":"2024-09-11T08:44:35.662980+0000","last_active":"2024-09-11T08:44:54.336448+0000","last_peered":"2024-09-11T08:44:54.336448+0000","last_clean":"2024-09-11T08:44:54.336448+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:44:54.336448+0000","last_undegraded":"2024-09-11T08:44:54.336448+0000","last_fullsized":"2024-09-11T08:44:54.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:20.735037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037767200000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":290,"num_read_kb":266,"num_write":22,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"63'161","reported_seq":508,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.162746+0000","last_change":"2024-09-11T08:44:35.668561+0000","last_active":"2024-09-11T08:44:54.162746+0000","last_peered":"2024-09-11T08:44:54.162746+0000","last_clean":"2024-09-11T08:44:54.162746+0000","last_became_active":"2024-09-11T08:44:11.733798+0000","last_became_peered":"2024-09-11T08:44:11.733798+0000","last_unstale":"2024-09-11T08:44:54.162746+0000","last_undegraded":"2024-09-11T08:44:54.162746+0000","last_fullsized":"2024-09-11T08:44:54.162746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:35:00.417002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096114499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":657,"num_read_kb":590,"num_write":79,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'90","reported_seq":380,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336492+0000","last_change":"2024-09-11T08:44:35.751698+0000","last_active":"2024-09-11T08:44:54.336492+0000","last_peered":"2024-09-11T08:44:54.336492+0000","last_clean":"2024-09-11T08:44:54.336492+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:44:54.336492+0000","last_undegraded":"2024-09-11T08:44:54.336492+0000","last_fullsized":"2024-09-11T08:44:54.336492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:28.421835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010508239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":306,"num_read_kb":258,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"63'164","reported_seq":607,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336462+0000","last_change":"2024-09-11T08:44:35.663067+0000","last_active":"2024-09-11T08:44:54.336462+0000","last_peered":"2024-09-11T08:44:54.336462+0000","last_clean":"2024-09-11T08:44:54.336462+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:44:54.336462+0000","last_undegraded":"2024-09-11T08:44:54.336462+0000","last_fullsized":"2024-09-11T08:44:54.336462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:27.366610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044274400000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":524,"num_write":70,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.066692+0000","last_change":"2024-09-11T08:44:02.988511+0000","last_active":"2024-09-11T08:44:54.066692+0000","last_peered":"2024-09-11T08:44:54.066692+0000","last_clean":"2024-09-11T08:44:54.066692+0000","last_became_active":"2024-09-11T08:44:02.988115+0000","last_became_peered":"2024-09-11T08:44:02.988115+0000","last_unstale":"2024-09-11T08:44:54.066692+0000","last_undegraded":"2024-09-11T08:44:54.066692+0000","last_fullsized":"2024-09-11T08:44:54.066692+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:51.312057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2846,"num_read_kb":2703,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":976,"ondisk_log_size":976,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36276,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:44:58.535 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:44:58.823 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:58.824 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:44:58.835 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-11T08:44:58.168443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2962,"num_read_kb":2803,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1093,"ondisk_log_size":1093,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":266416,"kb_used_data":8560,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708304,"statfs":{"total":773094113280,"available":772821303296,"internally_reserved":0,"allocated":8765440,"data_stored":4398846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.113290"},"pg_stats":[{"pgid":"2.7","version":"63'93","reported_seq":245,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336654+0000","last_change":"2024-09-11T08:44:35.753591+0000","last_active":"2024-09-11T08:44:54.336654+0000","last_peered":"2024-09-11T08:44:54.336654+0000","last_clean":"2024-09-11T08:44:54.336654+0000","last_became_active":"2024-09-11T08:44:28.934797+0000","last_became_peered":"2024-09-11T08:44:28.934797+0000","last_unstale":"2024-09-11T08:44:54.336654+0000","last_undegraded":"2024-09-11T08:44:54.336654+0000","last_fullsized":"2024-09-11T08:44:54.336654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:50.968529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027236169999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":154,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"63'135","reported_seq":411,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335927+0000","last_change":"2024-09-11T08:44:35.662727+0000","last_active":"2024-09-11T08:44:54.335927+0000","last_peered":"2024-09-11T08:44:54.335927+0000","last_clean":"2024-09-11T08:44:54.335927+0000","last_became_active":"2024-09-11T08:44:28.650336+0000","last_became_peered":"2024-09-11T08:44:28.650336+0000","last_unstale":"2024-09-11T08:44:54.335927+0000","last_undegraded":"2024-09-11T08:44:54.335927+0000","last_fullsized":"2024-09-11T08:44:54.335927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:47.097888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000391343,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":336,"num_write":69,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'73","reported_seq":251,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336525+0000","last_change":"2024-09-11T08:44:35.751095+0000","last_active":"2024-09-11T08:44:54.336525+0000","last_peered":"2024-09-11T08:44:54.336525+0000","last_clean":"2024-09-11T08:44:54.336525+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:44:54.336525+0000","last_undegraded":"2024-09-11T08:44:54.336525+0000","last_fullsized":"2024-09-11T08:44:54.336525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:09.418627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000448348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":154,"num_write":35,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"63'162","reported_seq":483,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335921+0000","last_change":"2024-09-11T08:44:35.662613+0000","last_active":"2024-09-11T08:44:54.335921+0000","last_peered":"2024-09-11T08:44:54.335921+0000","last_clean":"2024-09-11T08:44:54.335921+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:44:54.335921+0000","last_undegraded":"2024-09-11T08:44:54.335921+0000","last_fullsized":"2024-09-11T08:44:54.335921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037972899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":427,"num_read_kb":421,"num_write":107,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"63'98","reported_seq":460,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336448+0000","last_change":"2024-09-11T08:44:35.662980+0000","last_active":"2024-09-11T08:44:54.336448+0000","last_peered":"2024-09-11T08:44:54.336448+0000","last_clean":"2024-09-11T08:44:54.336448+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:44:54.336448+0000","last_undegraded":"2024-09-11T08:44:54.336448+0000","last_fullsized":"2024-09-11T08:44:54.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:20.735037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037767200000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":290,"num_read_kb":266,"num_write":22,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"63'161","reported_seq":508,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.162746+0000","last_change":"2024-09-11T08:44:35.668561+0000","last_active":"2024-09-11T08:44:54.162746+0000","last_peered":"2024-09-11T08:44:54.162746+0000","last_clean":"2024-09-11T08:44:54.162746+0000","last_became_active":"2024-09-11T08:44:11.733798+0000","last_became_peered":"2024-09-11T08:44:11.733798+0000","last_unstale":"2024-09-11T08:44:54.162746+0000","last_undegraded":"2024-09-11T08:44:54.162746+0000","last_fullsized":"2024-09-11T08:44:54.162746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:35:00.417002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096114499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":657,"num_read_kb":590,"num_write":79,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'90","reported_seq":380,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336492+0000","last_change":"2024-09-11T08:44:35.751698+0000","last_active":"2024-09-11T08:44:54.336492+0000","last_peered":"2024-09-11T08:44:54.336492+0000","last_clean":"2024-09-11T08:44:54.336492+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:44:54.336492+0000","last_undegraded":"2024-09-11T08:44:54.336492+0000","last_fullsized":"2024-09-11T08:44:54.336492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:28.421835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010508239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":306,"num_read_kb":258,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"63'164","reported_seq":607,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336462+0000","last_change":"2024-09-11T08:44:35.663067+0000","last_active":"2024-09-11T08:44:54.336462+0000","last_peered":"2024-09-11T08:44:54.336462+0000","last_clean":"2024-09-11T08:44:54.336462+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:44:54.336462+0000","last_undegraded":"2024-09-11T08:44:54.336462+0000","last_fullsized":"2024-09-11T08:44:54.336462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:27.366610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044274400000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":524,"num_write":70,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.066692+0000","last_change":"2024-09-11T08:44:02.988511+0000","last_active":"2024-09-11T08:44:54.066692+0000","last_peered":"2024-09-11T08:44:54.066692+0000","last_clean":"2024-09-11T08:44:54.066692+0000","last_became_active":"2024-09-11T08:44:02.988115+0000","last_became_peered":"2024-09-11T08:44:02.988115+0000","last_unstale":"2024-09-11T08:44:54.066692+0000","last_undegraded":"2024-09-11T08:44:54.066692+0000","last_fullsized":"2024-09-11T08:44:54.066692+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:51.312057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2846,"num_read_kb":2703,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":976,"ondisk_log_size":976,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36276,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:44:58.836 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T08:44:58.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:44:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:59.125 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:44:59.137 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-11T08:44:58.168443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2962,"num_read_kb":2803,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1093,"ondisk_log_size":1093,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":266416,"kb_used_data":8560,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708304,"statfs":{"total":773094113280,"available":772821303296,"internally_reserved":0,"allocated":8765440,"data_stored":4398846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.113290"},"pg_stats":[{"pgid":"2.7","version":"63'93","reported_seq":245,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336654+0000","last_change":"2024-09-11T08:44:35.753591+0000","last_active":"2024-09-11T08:44:54.336654+0000","last_peered":"2024-09-11T08:44:54.336654+0000","last_clean":"2024-09-11T08:44:54.336654+0000","last_became_active":"2024-09-11T08:44:28.934797+0000","last_became_peered":"2024-09-11T08:44:28.934797+0000","last_unstale":"2024-09-11T08:44:54.336654+0000","last_undegraded":"2024-09-11T08:44:54.336654+0000","last_fullsized":"2024-09-11T08:44:54.336654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:50.968529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027236169999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":154,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"63'135","reported_seq":411,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335927+0000","last_change":"2024-09-11T08:44:35.662727+0000","last_active":"2024-09-11T08:44:54.335927+0000","last_peered":"2024-09-11T08:44:54.335927+0000","last_clean":"2024-09-11T08:44:54.335927+0000","last_became_active":"2024-09-11T08:44:28.650336+0000","last_became_peered":"2024-09-11T08:44:28.650336+0000","last_unstale":"2024-09-11T08:44:54.335927+0000","last_undegraded":"2024-09-11T08:44:54.335927+0000","last_fullsized":"2024-09-11T08:44:54.335927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:47.097888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000391343,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":336,"num_write":69,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'73","reported_seq":251,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336525+0000","last_change":"2024-09-11T08:44:35.751095+0000","last_active":"2024-09-11T08:44:54.336525+0000","last_peered":"2024-09-11T08:44:54.336525+0000","last_clean":"2024-09-11T08:44:54.336525+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:44:54.336525+0000","last_undegraded":"2024-09-11T08:44:54.336525+0000","last_fullsized":"2024-09-11T08:44:54.336525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:09.418627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000448348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":154,"num_write":35,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"63'162","reported_seq":483,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335921+0000","last_change":"2024-09-11T08:44:35.662613+0000","last_active":"2024-09-11T08:44:54.335921+0000","last_peered":"2024-09-11T08:44:54.335921+0000","last_clean":"2024-09-11T08:44:54.335921+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:44:54.335921+0000","last_undegraded":"2024-09-11T08:44:54.335921+0000","last_fullsized":"2024-09-11T08:44:54.335921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037972899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":427,"num_read_kb":421,"num_write":107,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"63'98","reported_seq":460,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336448+0000","last_change":"2024-09-11T08:44:35.662980+0000","last_active":"2024-09-11T08:44:54.336448+0000","last_peered":"2024-09-11T08:44:54.336448+0000","last_clean":"2024-09-11T08:44:54.336448+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:44:54.336448+0000","last_undegraded":"2024-09-11T08:44:54.336448+0000","last_fullsized":"2024-09-11T08:44:54.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:20.735037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037767200000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":290,"num_read_kb":266,"num_write":22,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"63'161","reported_seq":508,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.162746+0000","last_change":"2024-09-11T08:44:35.668561+0000","last_active":"2024-09-11T08:44:54.162746+0000","last_peered":"2024-09-11T08:44:54.162746+0000","last_clean":"2024-09-11T08:44:54.162746+0000","last_became_active":"2024-09-11T08:44:11.733798+0000","last_became_peered":"2024-09-11T08:44:11.733798+0000","last_unstale":"2024-09-11T08:44:54.162746+0000","last_undegraded":"2024-09-11T08:44:54.162746+0000","last_fullsized":"2024-09-11T08:44:54.162746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:35:00.417002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096114499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":657,"num_read_kb":590,"num_write":79,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'90","reported_seq":380,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336492+0000","last_change":"2024-09-11T08:44:35.751698+0000","last_active":"2024-09-11T08:44:54.336492+0000","last_peered":"2024-09-11T08:44:54.336492+0000","last_clean":"2024-09-11T08:44:54.336492+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:44:54.336492+0000","last_undegraded":"2024-09-11T08:44:54.336492+0000","last_fullsized":"2024-09-11T08:44:54.336492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:28.421835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010508239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":306,"num_read_kb":258,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"63'164","reported_seq":607,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336462+0000","last_change":"2024-09-11T08:44:35.663067+0000","last_active":"2024-09-11T08:44:54.336462+0000","last_peered":"2024-09-11T08:44:54.336462+0000","last_clean":"2024-09-11T08:44:54.336462+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:44:54.336462+0000","last_undegraded":"2024-09-11T08:44:54.336462+0000","last_fullsized":"2024-09-11T08:44:54.336462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:27.366610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044274400000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":524,"num_write":70,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.066692+0000","last_change":"2024-09-11T08:44:02.988511+0000","last_active":"2024-09-11T08:44:54.066692+0000","last_peered":"2024-09-11T08:44:54.066692+0000","last_clean":"2024-09-11T08:44:54.066692+0000","last_became_active":"2024-09-11T08:44:02.988115+0000","last_became_peered":"2024-09-11T08:44:02.988115+0000","last_unstale":"2024-09-11T08:44:54.066692+0000","last_undegraded":"2024-09-11T08:44:54.066692+0000","last_fullsized":"2024-09-11T08:44:54.066692+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:51.312057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2846,"num_read_kb":2703,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":976,"ondisk_log_size":976,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36276,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:44:59.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T08:44:59.432 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:44:59.432 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"75755954-09e8-48d3-b9c1-83899efaff39","created":"2024-09-11T08:42:57.181838+0000","modified":"2024-09-11T08:44:54.063026+0000","last_up_change":"2024-09-11T08:43:09.343672+0000","last_in_change":"2024-09-11T08:44:27.636062+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:43:08.201248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T08:43:16.189979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":0.75,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac6b86cf-4ee8-49eb-a680-a350b1c9879c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6809","nonce":2540486849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6811","nonce":2540486849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6815","nonce":2540486849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2540486849},{"type":"v1","addr":"172.21.15.45:6813","nonce":2540486849}]},"public_addr":"172.21.15.45:6809/2540486849","cluster_addr":"172.21.15.45:6811/2540486849","heartbeat_back_addr":"172.21.15.45:6815/2540486849","heartbeat_front_addr":"172.21.15.45:6813/2540486849","state":["exists","up"]},{"osd":1,"uuid":"e27d776b-f77b-4111-a536-5f5d8030c696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6825","nonce":1040844445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6827","nonce":1040844445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6831","nonce":1040844445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1040844445},{"type":"v1","addr":"172.21.15.45:6829","nonce":1040844445}]},"public_addr":"172.21.15.45:6825/1040844445","cluster_addr":"172.21.15.45:6827/1040844445","heartbeat_back_addr":"172.21.15.45:6831/1040844445","heartbeat_front_addr":"172.21.15.45:6829/1040844445","state":["exists","up"]},{"osd":2,"uuid":"6b8e44f6-395b-447d-a5d3-c19a0a4be61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6817","nonce":1231172466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6819","nonce":1231172466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6823","nonce":1231172466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1231172466},{"type":"v1","addr":"172.21.15.45:6821","nonce":1231172466}]},"public_addr":"172.21.15.45:6817/1231172466","cluster_addr":"172.21.15.45:6819/1231172466","heartbeat_back_addr":"172.21.15.45:6823/1231172466","heartbeat_front_addr":"172.21.15.45:6821/1231172466","state":["exists","up"]},{"osd":3,"uuid":"1d1e1983-4c89-4cb4-a6b4-af244f941a7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6801","nonce":226138367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6803","nonce":226138367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6807","nonce":226138367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":226138367},{"type":"v1","addr":"172.21.15.45:6805","nonce":226138367}]},"public_addr":"172.21.15.45:6801/226138367","cluster_addr":"172.21.15.45:6803/226138367","heartbeat_back_addr":"172.21.15.45:6807/226138367","heartbeat_front_addr":"172.21.15.45:6805/226138367","state":["exists","up"]},{"osd":4,"uuid":"fd9239ee-3a74-4399-af2f-1e9a63a2f642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6801","nonce":374760845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6803","nonce":374760845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6807","nonce":374760845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":374760845},{"type":"v1","addr":"172.21.15.86:6805","nonce":374760845}]},"public_addr":"172.21.15.86:6801/374760845","cluster_addr":"172.21.15.86:6803/374760845","heartbeat_back_addr":"172.21.15.86:6807/374760845","heartbeat_front_addr":"172.21.15.86:6805/374760845","state":["exists","up"]},{"osd":5,"uuid":"cfff2555-5ad4-4629-a725-afadf9b6a96e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6825","nonce":498386186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6827","nonce":498386186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6831","nonce":498386186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":498386186},{"type":"v1","addr":"172.21.15.86:6829","nonce":498386186}]},"public_addr":"172.21.15.86:6825/498386186","cluster_addr":"172.21.15.86:6827/498386186","heartbeat_back_addr":"172.21.15.86:6831/498386186","heartbeat_front_addr":"172.21.15.86:6829/498386186","state":["exists","up"]},{"osd":6,"uuid":"4588b13e-af90-48a7-a10d-f7a185a902e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6809","nonce":1261869816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6811","nonce":1261869816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6815","nonce":1261869816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1261869816},{"type":"v1","addr":"172.21.15.86:6813","nonce":1261869816}]},"public_addr":"172.21.15.86:6809/1261869816","cluster_addr":"172.21.15.86:6811/1261869816","heartbeat_back_addr":"172.21.15.86:6815/1261869816","heartbeat_front_addr":"172.21.15.86:6813/1261869816","state":["exists","up"]},{"osd":7,"uuid":"12da1d21-9c59-4462-9d5a-61801dab3e4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6817","nonce":2975777507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6819","nonce":2975777507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6823","nonce":2975777507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2975777507},{"type":"v1","addr":"172.21.15.86:6821","nonce":2975777507}]},"public_addr":"172.21.15.86:6817/2975777507","cluster_addr":"172.21.15.86:6819/2975777507","heartbeat_back_addr":"172.21.15.86:6823/2975777507","heartbeat_front_addr":"172.21.15.86:6821/2975777507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.394370+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.813228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.425286+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.147591+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:07.119248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.653838+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.459308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:43:06.492589+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.0","osds":[2,0]},{"pgid":"2.1","osds":[0,1]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:45:00.445 INFO:tasks.ceph:Scrubbing osd.0 2024-09-11T08:45:00.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-11T08:45:00.586 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:45:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T08:45:00.586 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:45:00.597 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-11T08:45:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 0 to deep-scrub 2024-09-11T08:45:00.900 INFO:tasks.ceph:Scrubbing osd.1 2024-09-11T08:45:00.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-11T08:45:01.028 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:45:01.028 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T08:45:01.028 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:45:01.039 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-11T08:45:01.331 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 1 to deep-scrub 2024-09-11T08:45:01.344 INFO:tasks.ceph:Scrubbing osd.2 2024-09-11T08:45:01.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-11T08:45:01.471 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:45:01.471 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T08:45:01.471 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:45:01.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-11T08:45:01.775 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 2 to deep-scrub 2024-09-11T08:45:01.788 INFO:tasks.ceph:Scrubbing osd.3 2024-09-11T08:45:01.788 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-11T08:45:01.914 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:45:01.915 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T08:45:01.915 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:45:01.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-11T08:45:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 3 to deep-scrub 2024-09-11T08:45:02.232 INFO:tasks.ceph:Scrubbing osd.4 2024-09-11T08:45:02.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-11T08:45:02.362 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:45:02.362 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T08:45:02.362 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:45:02.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-11T08:45:02.663 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 4 to deep-scrub 2024-09-11T08:45:02.676 INFO:tasks.ceph:Scrubbing osd.5 2024-09-11T08:45:02.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-11T08:45:02.804 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:45:02.804 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "" 2024-09-11T08:45:02.804 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:45:02.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-11T08:45:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 5 to deep-scrub 2024-09-11T08:45:03.117 INFO:tasks.ceph:Scrubbing osd.6 2024-09-11T08:45:03.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-11T08:45:03.248 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:45:03.248 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T08:45:03.248 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:45:03.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-11T08:45:03.550 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 6 to deep-scrub 2024-09-11T08:45:03.563 INFO:tasks.ceph:Scrubbing osd.7 2024-09-11T08:45:03.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-11T08:45:03.691 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-11T08:45:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T08:45:03.691 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-11T08:45:03.703 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-11T08:45:03.994 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 7 to deep-scrub 2024-09-11T08:45:04.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:45:04.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:45:04.297 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:45:04.308 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-11T08:45:04.169489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2962,"num_read_kb":2803,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1093,"ondisk_log_size":1093,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":266416,"kb_used_data":8560,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708304,"statfs":{"total":773094113280,"available":772821303296,"internally_reserved":0,"allocated":8765440,"data_stored":4398846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.103124"},"pg_stats":[{"pgid":"2.7","version":"63'93","reported_seq":245,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336654+0000","last_change":"2024-09-11T08:44:35.753591+0000","last_active":"2024-09-11T08:44:54.336654+0000","last_peered":"2024-09-11T08:44:54.336654+0000","last_clean":"2024-09-11T08:44:54.336654+0000","last_became_active":"2024-09-11T08:44:28.934797+0000","last_became_peered":"2024-09-11T08:44:28.934797+0000","last_unstale":"2024-09-11T08:44:54.336654+0000","last_undegraded":"2024-09-11T08:44:54.336654+0000","last_fullsized":"2024-09-11T08:44:54.336654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:50.968529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027236169999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":154,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"63'135","reported_seq":411,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335927+0000","last_change":"2024-09-11T08:44:35.662727+0000","last_active":"2024-09-11T08:44:54.335927+0000","last_peered":"2024-09-11T08:44:54.335927+0000","last_clean":"2024-09-11T08:44:54.335927+0000","last_became_active":"2024-09-11T08:44:28.650336+0000","last_became_peered":"2024-09-11T08:44:28.650336+0000","last_unstale":"2024-09-11T08:44:54.335927+0000","last_undegraded":"2024-09-11T08:44:54.335927+0000","last_fullsized":"2024-09-11T08:44:54.335927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:47.097888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000391343,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":336,"num_write":69,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'73","reported_seq":251,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336525+0000","last_change":"2024-09-11T08:44:35.751095+0000","last_active":"2024-09-11T08:44:54.336525+0000","last_peered":"2024-09-11T08:44:54.336525+0000","last_clean":"2024-09-11T08:44:54.336525+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:44:54.336525+0000","last_undegraded":"2024-09-11T08:44:54.336525+0000","last_fullsized":"2024-09-11T08:44:54.336525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:09.418627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000448348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":154,"num_write":35,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"63'162","reported_seq":483,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.335921+0000","last_change":"2024-09-11T08:44:35.662613+0000","last_active":"2024-09-11T08:44:54.335921+0000","last_peered":"2024-09-11T08:44:54.335921+0000","last_clean":"2024-09-11T08:44:54.335921+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:44:54.335921+0000","last_undegraded":"2024-09-11T08:44:54.335921+0000","last_fullsized":"2024-09-11T08:44:54.335921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037972899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":427,"num_read_kb":421,"num_write":107,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"63'98","reported_seq":460,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336448+0000","last_change":"2024-09-11T08:44:35.662980+0000","last_active":"2024-09-11T08:44:54.336448+0000","last_peered":"2024-09-11T08:44:54.336448+0000","last_clean":"2024-09-11T08:44:54.336448+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:44:54.336448+0000","last_undegraded":"2024-09-11T08:44:54.336448+0000","last_fullsized":"2024-09-11T08:44:54.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:20.735037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037767200000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":290,"num_read_kb":266,"num_write":22,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"63'161","reported_seq":508,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.162746+0000","last_change":"2024-09-11T08:44:35.668561+0000","last_active":"2024-09-11T08:44:54.162746+0000","last_peered":"2024-09-11T08:44:54.162746+0000","last_clean":"2024-09-11T08:44:54.162746+0000","last_became_active":"2024-09-11T08:44:11.733798+0000","last_became_peered":"2024-09-11T08:44:11.733798+0000","last_unstale":"2024-09-11T08:44:54.162746+0000","last_undegraded":"2024-09-11T08:44:54.162746+0000","last_fullsized":"2024-09-11T08:44:54.162746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:35:00.417002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096114499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":657,"num_read_kb":590,"num_write":79,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'90","reported_seq":380,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336492+0000","last_change":"2024-09-11T08:44:35.751698+0000","last_active":"2024-09-11T08:44:54.336492+0000","last_peered":"2024-09-11T08:44:54.336492+0000","last_clean":"2024-09-11T08:44:54.336492+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:44:54.336492+0000","last_undegraded":"2024-09-11T08:44:54.336492+0000","last_fullsized":"2024-09-11T08:44:54.336492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:28.421835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010508239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":306,"num_read_kb":258,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"63'164","reported_seq":607,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.336462+0000","last_change":"2024-09-11T08:44:35.663067+0000","last_active":"2024-09-11T08:44:54.336462+0000","last_peered":"2024-09-11T08:44:54.336462+0000","last_clean":"2024-09-11T08:44:54.336462+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:44:54.336462+0000","last_undegraded":"2024-09-11T08:44:54.336462+0000","last_fullsized":"2024-09-11T08:44:54.336462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:16.369891+0000","last_clean_scrub_stamp":"2024-09-11T08:43:16.369891+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:27.366610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044274400000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":524,"num_write":70,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:44:54.066692+0000","last_change":"2024-09-11T08:44:02.988511+0000","last_active":"2024-09-11T08:44:54.066692+0000","last_peered":"2024-09-11T08:44:54.066692+0000","last_clean":"2024-09-11T08:44:54.066692+0000","last_became_active":"2024-09-11T08:44:02.988115+0000","last_became_peered":"2024-09-11T08:44:02.988115+0000","last_unstale":"2024-09-11T08:44:54.066692+0000","last_undegraded":"2024-09-11T08:44:54.066692+0000","last_fullsized":"2024-09-11T08:44:54.066692+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:08.334982+0000","last_clean_scrub_stamp":"2024-09-11T08:43:08.334982+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:51.312057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2846,"num_read_kb":2703,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":976,"ondisk_log_size":976,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32696,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509467,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36276,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:45:04.312 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-11T08:43:16.369891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=16, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.312 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-11T08:43:16.369891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=16, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.312 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-11T08:43:16.369891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=16, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.313 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-11T08:43:16.369891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=16, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.313 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-11T08:43:16.369891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=16, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.313 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-11T08:43:16.369891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=16, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.313 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-11T08:43:16.369891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=16, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.313 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-11T08:43:16.369891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=16, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.313 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-11T08:43:08.334982+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=43, tm_sec=8, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=8, tm_min=44, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T08:45:04.313 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-11T08:45:24.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T08:45:24.618 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-11T08:45:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-11T08:45:24.631 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-11T08:45:24.174170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2962,"num_read_kb":2803,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1097,"ondisk_log_size":1097,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":266368,"kb_used_data":8560,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708352,"statfs":{"total":773094113280,"available":772821352448,"internally_reserved":0,"allocated":8765440,"data_stored":4398846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002648"},"pg_stats":[{"pgid":"2.7","version":"63'93","reported_seq":254,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:04.279526+0000","last_change":"2024-09-11T08:45:04.279552+0000","last_active":"2024-09-11T08:45:04.279526+0000","last_peered":"2024-09-11T08:45:04.279526+0000","last_clean":"2024-09-11T08:45:04.279526+0000","last_became_active":"2024-09-11T08:44:28.934797+0000","last_became_peered":"2024-09-11T08:44:28.934797+0000","last_unstale":"2024-09-11T08:45:04.279526+0000","last_undegraded":"2024-09-11T08:45:04.279526+0000","last_fullsized":"2024-09-11T08:45:04.279526+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'93","last_scrub_stamp":"2024-09-11T08:45:04.279427+0000","last_deep_scrub":"63'93","last_deep_scrub_stamp":"2024-09-11T08:45:04.279427+0000","last_clean_scrub_stamp":"2024-09-11T08:45:04.279427+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:33:02.585190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027236169999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":154,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"63'135","reported_seq":420,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:02.612506+0000","last_change":"2024-09-11T08:45:02.612529+0000","last_active":"2024-09-11T08:45:02.612506+0000","last_peered":"2024-09-11T08:45:02.612506+0000","last_clean":"2024-09-11T08:45:02.612506+0000","last_became_active":"2024-09-11T08:44:28.650336+0000","last_became_peered":"2024-09-11T08:44:28.650336+0000","last_unstale":"2024-09-11T08:45:02.612506+0000","last_undegraded":"2024-09-11T08:45:02.612506+0000","last_fullsized":"2024-09-11T08:45:02.612506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'135","last_scrub_stamp":"2024-09-11T08:45:02.612426+0000","last_deep_scrub":"63'135","last_deep_scrub_stamp":"2024-09-11T08:45:02.612426+0000","last_clean_scrub_stamp":"2024-09-11T08:45:02.612426+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:30:02.977731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000391343,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":336,"num_write":69,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'73","reported_seq":260,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:04.084177+0000","last_change":"2024-09-11T08:45:04.084207+0000","last_active":"2024-09-11T08:45:04.084177+0000","last_peered":"2024-09-11T08:45:04.084177+0000","last_clean":"2024-09-11T08:45:04.084177+0000","last_became_active":"2024-09-11T08:43:17.384037+0000","last_became_peered":"2024-09-11T08:43:17.384037+0000","last_unstale":"2024-09-11T08:45:04.084177+0000","last_undegraded":"2024-09-11T08:45:04.084177+0000","last_fullsized":"2024-09-11T08:45:04.084177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'73","last_scrub_stamp":"2024-09-11T08:45:04.084030+0000","last_deep_scrub":"63'73","last_deep_scrub_stamp":"2024-09-11T08:45:04.084030+0000","last_clean_scrub_stamp":"2024-09-11T08:45:04.084030+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:09.570383+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000448348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":154,"num_write":35,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"63'162","reported_seq":492,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:01.629435+0000","last_change":"2024-09-11T08:45:01.629460+0000","last_active":"2024-09-11T08:45:01.629435+0000","last_peered":"2024-09-11T08:45:01.629435+0000","last_clean":"2024-09-11T08:45:01.629435+0000","last_became_active":"2024-09-11T08:43:46.863626+0000","last_became_peered":"2024-09-11T08:43:46.863626+0000","last_unstale":"2024-09-11T08:45:01.629435+0000","last_undegraded":"2024-09-11T08:45:01.629435+0000","last_fullsized":"2024-09-11T08:45:01.629435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'162","last_scrub_stamp":"2024-09-11T08:45:01.629355+0000","last_deep_scrub":"63'162","last_deep_scrub_stamp":"2024-09-11T08:45:01.629355+0000","last_clean_scrub_stamp":"2024-09-11T08:45:01.629355+0000","objects_scrubbed":1,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:17.863542+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00037972899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":427,"num_read_kb":421,"num_write":107,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"81'100","reported_seq":471,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:04.676309+0000","last_change":"2024-09-11T08:45:04.676363+0000","last_active":"2024-09-11T08:45:04.676309+0000","last_peered":"2024-09-11T08:45:04.676309+0000","last_clean":"2024-09-11T08:45:04.676309+0000","last_became_active":"2024-09-11T08:43:46.491187+0000","last_became_peered":"2024-09-11T08:43:46.491187+0000","last_unstale":"2024-09-11T08:45:04.676309+0000","last_undegraded":"2024-09-11T08:45:04.676309+0000","last_fullsized":"2024-09-11T08:45:04.676309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'100","last_scrub_stamp":"2024-09-11T08:45:04.676196+0000","last_deep_scrub":"81'100","last_deep_scrub_stamp":"2024-09-11T08:45:04.676196+0000","last_clean_scrub_stamp":"2024-09-11T08:45:04.676196+0000","objects_scrubbed":2,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:31.887504+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00037767200000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":290,"num_read_kb":266,"num_write":22,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"63'161","reported_seq":517,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:01.008842+0000","last_change":"2024-09-11T08:45:01.008864+0000","last_active":"2024-09-11T08:45:01.008842+0000","last_peered":"2024-09-11T08:45:01.008842+0000","last_clean":"2024-09-11T08:45:01.008842+0000","last_became_active":"2024-09-11T08:44:11.733798+0000","last_became_peered":"2024-09-11T08:44:11.733798+0000","last_unstale":"2024-09-11T08:45:01.008842+0000","last_undegraded":"2024-09-11T08:45:01.008842+0000","last_fullsized":"2024-09-11T08:45:01.008842+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'161","last_scrub_stamp":"2024-09-11T08:45:01.008762+0000","last_deep_scrub":"63'161","last_deep_scrub_stamp":"2024-09-11T08:45:01.008762+0000","last_clean_scrub_stamp":"2024-09-11T08:45:01.008762+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:27:22.190492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096114499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":657,"num_read_kb":590,"num_write":79,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'90","reported_seq":390,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:05.058497+0000","last_change":"2024-09-11T08:45:05.058497+0000","last_active":"2024-09-11T08:45:05.058497+0000","last_peered":"2024-09-11T08:45:05.058497+0000","last_clean":"2024-09-11T08:45:05.058497+0000","last_became_active":"2024-09-11T08:43:47.895675+0000","last_became_peered":"2024-09-11T08:43:47.895675+0000","last_unstale":"2024-09-11T08:45:05.058497+0000","last_undegraded":"2024-09-11T08:45:05.058497+0000","last_fullsized":"2024-09-11T08:45:05.058497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'90","last_scrub_stamp":"2024-09-11T08:45:05.058339+0000","last_deep_scrub":"63'90","last_deep_scrub_stamp":"2024-09-11T08:45:05.058339+0000","last_clean_scrub_stamp":"2024-09-11T08:45:05.058339+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:27:20.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010508239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":306,"num_read_kb":258,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"63'164","reported_seq":616,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:03.639397+0000","last_change":"2024-09-11T08:45:03.639443+0000","last_active":"2024-09-11T08:45:03.639397+0000","last_peered":"2024-09-11T08:45:03.639397+0000","last_clean":"2024-09-11T08:45:03.639397+0000","last_became_active":"2024-09-11T08:43:17.383137+0000","last_became_peered":"2024-09-11T08:43:17.383137+0000","last_unstale":"2024-09-11T08:45:03.639397+0000","last_undegraded":"2024-09-11T08:45:03.639397+0000","last_fullsized":"2024-09-11T08:45:03.639397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'164","last_scrub_stamp":"2024-09-11T08:45:03.639274+0000","last_deep_scrub":"63'164","last_deep_scrub_stamp":"2024-09-11T08:45:03.639274+0000","last_clean_scrub_stamp":"2024-09-11T08:45:03.639274+0000","objects_scrubbed":1,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:54:32.446232+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044274400000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":524,"num_write":70,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"81'119","reported_seq":250,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:45:02.300603+0000","last_change":"2024-09-11T08:45:02.300630+0000","last_active":"2024-09-11T08:45:02.300603+0000","last_peered":"2024-09-11T08:45:02.300603+0000","last_clean":"2024-09-11T08:45:02.300603+0000","last_became_active":"2024-09-11T08:44:02.988115+0000","last_became_peered":"2024-09-11T08:44:02.988115+0000","last_unstale":"2024-09-11T08:45:02.300603+0000","last_undegraded":"2024-09-11T08:45:02.300603+0000","last_fullsized":"2024-09-11T08:45:02.300603+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'119","last_scrub_stamp":"2024-09-11T08:45:02.300524+0000","last_deep_scrub":"81'119","last_deep_scrub_stamp":"2024-09-11T08:45:02.300524+0000","last_clean_scrub_stamp":"2024-09-11T08:45:02.300524+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:46:12.347657+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2846,"num_read_kb":2703,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":978,"ondisk_log_size":978,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335568,"statfs":{"total":96636764160,"available":96599621632,"internally_reserved":0,"allocated":950272,"data_stored":402278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36848,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334992,"statfs":{"total":96636764160,"available":96599031808,"internally_reserved":0,"allocated":1540096,"data_stored":992627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36276,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335564,"statfs":{"total":96636764160,"available":96599617536,"internally_reserved":0,"allocated":946176,"data_stored":402259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:45:24.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-11T08:45:24.937 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-11T08:45:24.937 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-11T08:45:24.937 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T08:45:24.938 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:24.982 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T08:45:24.982 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T08:45:24.982 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.025 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T08:45:25.025 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T08:45:25.025 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.062 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T08:45:25.062 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-11T08:45:25.062 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.133 INFO:tasks.ceph.osd.3:Stopped 2024-09-11T08:45:25.133 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-11T08:45:25.133 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.173 INFO:tasks.ceph.osd.4:Stopped 2024-09-11T08:45:25.173 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-11T08:45:25.174 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.251 INFO:tasks.ceph.osd.5:Stopped 2024-09-11T08:45:25.251 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-11T08:45:25.251 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.325 INFO:tasks.ceph.osd.6:Stopped 2024-09-11T08:45:25.325 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-11T08:45:25.325 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.397 INFO:tasks.ceph.osd.7:Stopped 2024-09-11T08:45:25.398 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-11T08:45:25.398 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-11T08:45:25.398 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.422 INFO:tasks.ceph.mgr.y:Stopped 2024-09-11T08:45:25.422 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-11T08:45:25.422 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.467 INFO:tasks.ceph.mgr.x:Stopped 2024-09-11T08:45:25.467 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-11T08:45:25.467 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-11T08:45:25.467 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.478 INFO:tasks.ceph.mon.a:Stopped 2024-09-11T08:45:25.479 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-11T08:45:25.479 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.530 INFO:tasks.ceph.mon.c:Stopped 2024-09-11T08:45:25.530 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-11T08:45:25.530 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T08:45:25.543 INFO:tasks.ceph.mon.b:Stopped 2024-09-11T08:45:25.543 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-11T08:45:25.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-11T08:45:25.605 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:45:25.606 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-11T08:45:25.833 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:45:25.833 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-11T08:45:25.914 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:45:25.914 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-11T08:45:25.988 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:45:25.988 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-11T08:45:26.039 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:45:26.039 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-11T08:45:26.167 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:45:26.167 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-11T08:45:26.255 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:45:26.255 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-11T08:45:26.305 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:45:26.306 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-11T08:45:26.353 INFO:tasks.ceph:Archiving mon data... 2024-09-11T08:45:26.354 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/data/mon.a.tgz 2024-09-11T08:45:26.354 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-11T08:45:26.370 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.WLuFAk9rSa 2024-09-11T08:45:26.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.WLuFAk9rSa 2024-09-11T08:45:26.516 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.WLuFAk9rSa 2024-09-11T08:45:26.600 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.WLuFAk9rSa is 281KB 2024-09-11T08:45:26.695 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.WLuFAk9rSa 2024-09-11T08:45:26.711 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/data/mon.c.tgz 2024-09-11T08:45:26.711 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-11T08:45:26.765 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.b65t6v9d9w 2024-09-11T08:45:26.766 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.b65t6v9d9w 2024-09-11T08:45:26.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.b65t6v9d9w 2024-09-11T08:45:26.988 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.b65t6v9d9w is 388KB 2024-09-11T08:45:27.084 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.b65t6v9d9w 2024-09-11T08:45:27.100 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/data/mon.b.tgz 2024-09-11T08:45:27.100 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-09-11T08:45:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.uw1LXMIvRB 2024-09-11T08:45:27.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.uw1LXMIvRB 2024-09-11T08:45:27.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.uw1LXMIvRB 2024-09-11T08:45:27.356 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.uw1LXMIvRB is 388KB 2024-09-11T08:45:27.451 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.uw1LXMIvRB 2024-09-11T08:45:27.466 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-11T08:45:27.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T08:45:27.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T08:45:27.550 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T08:45:27.584 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T08:45:27.584 INFO:tasks.ceph:Archiving crash dumps... 2024-09-11T08:45:27.586 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/remote/smithi045/crash 2024-09-11T08:45:27.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T08:45:27.617 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/remote/smithi086/crash 2024-09-11T08:45:27.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T08:45:27.648 INFO:tasks.ceph:Compressing logs... 2024-09-11T08:45:27.648 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:45:27.662 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:45:27.685 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-11T08:45:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33115.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33286.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33115.log: -- /var/log/ceph/ceph-osd.7.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33115.log.gz 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33246.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33155.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33075.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33035.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33206.log 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.33246.log: /var/log/ceph/ceph-client.admin.33155.log: /var/log/ceph/ceph-osd.4.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33286.log.gz 2024-09-11T08:45:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33246.log.gz 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33155.log.gz 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr: 26.4% -- replaced with /var/log/ceph/ceph-client.admin.33035.log.gz 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.33075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33075.log.gz 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33206.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33206.log.gz 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33326.log 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33326.log: 84.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T08:45:27.728 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33326.log.gz 2024-09-11T08:45:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-11T08:45:27.776 INFO:teuthology.orchestra.run.smithi086.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-11T08:45:27.793 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27113.log 2024-09-11T08:45:27.793 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48226.log 2024-09-11T08:45:27.793 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph.tmp-client.admin.27113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38298.log 2024-09-11T08:45:27.793 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27113.log.gz 2024-09-11T08:45:27.793 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41496.log 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48226.log.gz 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51290.log 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38298.log.gz 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41496.log.gz 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51370.log 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51290.log.gz 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52196.log 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38923.log 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51370.log.gz 2024-09-11T08:45:27.794 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50692.log 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52196.log.gz 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33273.log 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38923.log.gz 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42794.log 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50692.log.gz 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49401.log 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33273.log.gz 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47361.log 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42794.log.gz 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49401.log.gz 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48194.log 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37971.log 2024-09-11T08:45:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47361.log.gz 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37963.log 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48194.log.gz 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49363.log 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37971.log.gz 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37963.log.gz 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.49363.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48122.log 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.49363.log.gz 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51902.log 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52154.log 2024-09-11T08:45:27.796 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48122.log.gz 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32948.log 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51902.log.gz 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51372.log 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52154.log.gz 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41756.log 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32948.log.gz 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37654.log 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51372.log.gz 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47174.log 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41756.log.gz 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45339.log 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37654.log.gz 2024-09-11T08:45:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47174.log.gz 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39049.log 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45339.log.gz 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37956.log 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49128.log 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39049.log.gz 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50306.log 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37956.log.gz 2024-09-11T08:45:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49128.log.gz 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51566.log 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49088.log 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50306.log.gz 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38881.log 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51566.log.gz 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38297.log 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.49088.log.gz 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49970.log 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38881.log.gz 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37965.log 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38297.log.gz 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52028.log 2024-09-11T08:45:27.799 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49970.log.gz 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47642.log 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37965.log.gz 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52028.log.gz 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43701.log 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47642.log.gz 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43619.log 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44000.log 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43701.log.gz 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52280.log 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43619.log.gz 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44000.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-11T08:45:27.800 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44000.log.gz 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52280.log.gz 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52406.log 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48442.log 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52406.log.gz 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52112.log 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48442.log.gz 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39668.log 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-09-11T08:45:27.801 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52112.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.52112.log.gz 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48592.log 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49268.log 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39668.log.gz 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41071.log 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48592.log.gz 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50264.log 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49268.log.gz 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47435.log 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41071.log.gz 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38304.log 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50264.log.gz 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47435.log.gz 2024-09-11T08:45:27.802 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43485.log 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38304.log.gz 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48553.log 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47528.log 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43485.log.gz 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48553.log.gz 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38306.log 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38523.log 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47528.log.gz 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38591.log 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38306.log.gz 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38307.log 2024-09-11T08:45:27.803 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38523.log.gz 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38591.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38591.log.gz 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50954.log 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37486.log 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38307.log.gz 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50954.log.gz 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50390.log 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47157.log 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37486.log.gz 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50390.log.gz 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47548.log 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47157.log.gz 2024-09-11T08:45:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47548.log.gz 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51371.log 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51371.log.gz 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44835.log 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50432.log 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44835.log.gz 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49540.log 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42675.log 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50432.log.gz 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52448.log 2024-09-11T08:45:27.805 INFO:teuthology.orchestra.run.smithi045.stderr: 54.4% -- replaced with /var/log/ceph/ceph-client.admin.49540.log.gz 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48745.log 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42675.log.gz 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52448.log.gz 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45396.log 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44477.log 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48745.log.gz 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45396.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45396.log.gz 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.37964.log 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50942.log 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44477.log.gz 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52490.log 0.0% 2024-09-11T08:45:27.806 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37964.log.gz 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.50942.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41135.log 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50942.log.gz 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47021.log 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52490.log.gz 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39289.log 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41135.log.gz 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44199.log 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47021.log.gz 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47284.log 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39289.log.gz 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48142.log 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44199.log.gz 2024-09-11T08:45:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43137.log 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47284.log.gz 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48142.log.gz 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43256.log 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43137.log.gz 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43256.log.gz 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45735.log 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48410.log 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45735.log.gz 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49300.log 2024-09-11T08:45:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50697.log 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48410.log.gz 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49300.log.gz 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40549.log 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49671.log 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50697.log.gz 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40549.log.gz 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39290.log 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50180.log 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49671.log.gz 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39290.log.gz 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40427.log 2024-09-11T08:45:27.809 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50180.log.gz 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47140.log 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40248.log 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40427.log.gz 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47140.log.gz 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46275.log 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40248.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40248.log.gz 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46275.log.gz 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52574.log 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40973.log 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52574.log.gz 2024-09-11T08:45:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47610.log 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51032.log 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40973.log.gz 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39091.log 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr: 6.4% -- replaced with /var/log/ceph/ceph-client.admin.47610.log.gz 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51032.log.gz 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38839.log 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49332.log 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39091.log.gz 2024-09-11T08:45:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38839.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38839.log.gz 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37969.log 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51776.log 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49332.log.gz 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45116.log 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37969.log.gz 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48869.log 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51776.log.gz 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48301.log 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45116.log.gz 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51860.log 2024-09-11T08:45:27.812 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48869.log.gz 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51986.log 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48301.log.gz 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49483.log 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51860.log.gz 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52532.log 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51986.log.gz 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44904.log 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49483.log.gz 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33349.log 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52532.log.gz 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44904.log.gz 2024-09-11T08:45:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48536.log 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33349.log.gz 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48712.log 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48356.log 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48536.log.gz 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40611.log 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48712.log.gz 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38593.log 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr: 25.0% -- replaced with /var/log/ceph/ceph-client.admin.48356.log.gz 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38305.log 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40611.log.gz 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41579.log 2024-09-11T08:45:27.814 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38593.log.gz 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41279.log 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38305.log.gz 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47589.log 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41579.log.gz 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47078.log 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41279.log.gz 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38167.log 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47589.log.gz 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47078.log.gz 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46612.log 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38167.log.gz 2024-09-11T08:45:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48492.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr: 51.4% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46816.log 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46612.log.gz 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37966.log 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46816.log.gz 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47788.log 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37966.log.gz 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44614.log 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47788.log.gz 2024-09-11T08:45:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43418.log 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41177.log 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44614.log.gz 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47703.log 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43418.log.gz 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44097.log 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41177.log.gz 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47264.log 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47703.log.gz 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44097.log.gz 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47488.log 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47264.log.gz 2024-09-11T08:45:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50222.log 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr: 5.5% -- replaced with /var/log/ceph/ceph-client.admin.47488.log.gz 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50222.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50222.log.gz 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47901.log 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47247.log: /var/log/ceph/ceph-client.admin.51944.log 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47901.log: 4.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.47901.log.gz -5 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47382.log 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48630.log 2024-09-11T08:45:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51944.log.gz 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47382.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44797.log 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47382.log.gz 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48630.log.gz 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44797.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44797.log.gz -5 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50054.log 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38660.log 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50054.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.50054.log.gz -5 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51033.log 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50012.log 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38660.log.gz 2024-09-11T08:45:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49718.log 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51033.log.gz 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37573.log 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50012.log.gz 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40832.log 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49718.log.gz 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37573.log.gz 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40832.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40832.log.gz 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr: -- /var/log/ceph/ceph-client.admin.51734.log 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39292.log 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51734.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51734.log.gz 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48670.log 2024-09-11T08:45:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52238.log 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39292.log.gz 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47344.log 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48670.log.gz 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40084.log 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52238.log.gz 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41857.log 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47344.log.gz 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40670.log 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40084.log.gz 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48257.log 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41857.log.gz 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48081.log 2024-09-11T08:45:27.821 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40670.log.gz 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51230.log 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr: 51.6% -- replaced with /var/log/ceph/ceph-client.admin.48257.log.gz 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45776.log 53.0% -- replaced with /var/log/ceph/ceph-client.admin.48081.log.gz 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51230.log.gz 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47858.log 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45776.log.gz 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41051.log 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47858.log.gz 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41051.log.gz 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51369.log 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48808.log 2024-09-11T08:45:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51369.log.gz 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41974.log 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48808.log: 51.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40529.log 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48808.log.gz 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45934.log 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41974.log.gz 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38663.log 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40529.log: /var/log/ceph/ceph-client.admin.45934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51031.log 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40529.log.gz 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45934.log.gz 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38663.log: /var/log/ceph/ceph-client.admin.50952.log 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38663.log.gz 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51031.log.gz 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-09-11T08:45:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log/var/log/ceph/ceph-client.admin.50952.log: 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50952.log.gz 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38965.log 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47508.log: 5.5% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48777.log 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45017.log: /var/log/ceph/ceph-client.admin.38965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38965.log.gz 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48777.log.gz 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50694.log 2024-09-11T08:45:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42356.log 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52322.log 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50694.log.gz 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51650.log 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42356.log: /var/log/ceph/ceph-client.admin.52322.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42356.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52322.log.gz 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51650.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51650.log.gz 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51608.log 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49225.log 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 2024-09-11T08:45:27.825 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51608.log.gz 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49225.log.gz 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45538.log 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.47928.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gz 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph.log 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45538.log.gz 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39153.log 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47119.log 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50691.log 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39153.log.gz 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47119.log.gz 2024-09-11T08:45:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50696.log 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49507.log 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50691.log.gz 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50693.log 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50696.log.gz 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50138.log 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49507.log.gz 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50693.log: -- /var/log/ceph/ceph-client.admin.39007.log 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50693.log.gz 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50138.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50138.log.gz 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47204.log 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39007.log.gz 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49760.log 2024-09-11T08:45:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47204.log.gz 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49145.log 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38303.log 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49760.log: /var/log/ceph/ceph-client.admin.49145.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.48910.log 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49760.log.gz 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49145.log.gz 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38303.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38303.log.gz 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50348.log 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48910.log.gz 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46116.log 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50348.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50348.log.gz 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37789.log 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46116.log.gz 2024-09-11T08:45:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50690.log 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37789.log.gz 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50516.log 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50690.log: -- /var/log/ceph/ceph-client.admin.47306.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50690.log.gz 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50516.log.gz 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47724.log 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47306.log.gz 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50096.log 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47724.log: 4.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51818.log 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47724.log.gz 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49802.log 2024-09-11T08:45:27.829 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50096.log.gz 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51692.log 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51818.log: /var/log/ceph/ceph-client.admin.49802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51818.log.gz 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44737.log 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49802.log.gz 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42413.log 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51692.log.gz 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48666.log 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44737.log: /var/log/ceph/ceph-client.admin.42413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49423.log 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42413.log.gz 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44737.log.gz 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48666.log.gz 2024-09-11T08:45:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39725.log 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49423.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42595.log 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49423.log.gz 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49905.log 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39725.log.gz 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38584.log 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42595.log: /var/log/ceph/ceph-client.admin.49905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42057.log 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42595.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49905.log.gz 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38584.log.gz 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51289.log 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42057.log.gz 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50474.log 2024-09-11T08:45:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51289.log.gz 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37747.log 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50474.log.gz 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47857.log 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37747.log.gz 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52364.log 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47857.log.gz 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49185.log 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52364.log.gz 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49384.log 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49185.log: 25.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42854.log 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49185.log.gz 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47687.log 2024-09-11T08:45:27.832 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49384.log.gz 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50698.log 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42854.log: /var/log/ceph/ceph-client.admin.47687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42854.log.gz 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47456.log 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47687.log.gz 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50698.log.gz 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51034.log 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47456.log.gz 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48335.log: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52614.log 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51034.log: /var/log/ceph/ceph-client.admin.52614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38503.log 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51034.log.gz 2024-09-11T08:45:27.833 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52614.log.gz 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log/var/log/ceph/ceph-client.admin.42153.log: 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38503.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38503.log.gz 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52070.log 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39967.log 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48056.log 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52070.log.gz 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39967.log.gz 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40064.log 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51288.log 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48056.log: /var/log/ceph/ceph-client.admin.40064.log: 51.8% -- replaced with /var/log/ceph/ceph-client.admin.48056.log.gz 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40064.log.gz 2024-09-11T08:45:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49029.log 2024-09-11T08:45:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38665.log 2024-09-11T08:45:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51288.log.gz 2024-09-11T08:45:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49029.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49455.log 2024-09-11T08:45:27.835 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49029.log.gz 2024-09-11T08:45:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38665.log.gz 2024-09-11T08:45:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47225.log 2024-09-11T08:45:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49455.log.gz/var/log/ceph/ceph-client.admin.47225.log: 2024-09-11T08:45:27.835 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47225.log.gz 2024-09-11T08:45:27.836 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-11T08:45:27.836 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-11T08:45:27.836 INFO:teuthology.orchestra.run.smithi086.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-11T08:45:27.838 INFO:teuthology.orchestra.run.smithi045.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-11T08:45:27.839 INFO:teuthology.orchestra.run.smithi045.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-11T08:45:27.885 INFO:teuthology.orchestra.run.smithi086.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-11T08:45:27.892 INFO:teuthology.orchestra.run.smithi086.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-11T08:45:27.894 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-09-11T08:45:27.894 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m0.188s 2024-09-11T08:45:27.894 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m0.630s 2024-09-11T08:45:27.894 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.043s 2024-09-11T08:45:27.913 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-11T08:45:28.052 INFO:teuthology.orchestra.run.smithi045.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-11T08:45:28.906 INFO:teuthology.orchestra.run.smithi045.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-11T08:45:28.908 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-11T08:45:28.908 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m1.233s 2024-09-11T08:45:28.908 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m2.025s 2024-09-11T08:45:28.908 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.210s 2024-09-11T08:45:28.909 INFO:tasks.ceph:Archiving logs... 2024-09-11T08:45:28.909 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/remote/smithi045/log 2024-09-11T08:45:28.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:45:29.414 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/remote/smithi086/log 2024-09-11T08:45:29.415 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:45:29.541 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T08:45:29.555 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T08:45:29.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:45:29.558 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:45:29.639 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T08:45:29.647 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T08:45:29.648 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:45:29.650 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:45:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:45:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:45:29.669 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:45:29.669 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-11T08:45:29.669 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -1977us[-1898us] +/- 55ms 2024-09-11T08:45:29.669 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 +1712us[+1791us] +/- 77ms 2024-09-11T08:45:29.669 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +999us[+1078us] +/- 63ms 2024-09-11T08:45:29.669 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:45:29.671 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:45:29.671 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-09-11T08:45:29.671 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 -1968us[-1858us] +/- 55ms 2024-09-11T08:45:29.671 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 +1713us[+1713us] +/- 77ms 2024-09-11T08:45:29.671 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +1010us[+1120us] +/- 63ms 2024-09-11T08:45:29.671 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:45:29.672 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T08:45:29.722 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T08:45:29.723 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T08:45:29.731 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:45:29.807 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:45:29.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T08:45:29.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.912 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T08:45:29.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.942 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T08:45:29.951 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T08:45:29.959 INFO:teuthology.task.internal:Duration was 571.629040 seconds 2024-09-11T08:45:29.959 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T08:45:29.967 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T08:45:29.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:45:29.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:45:30.009 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:45:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:45:30.425 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T08:45:30.425 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-09-11T08:45:30.425 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:45:30.488 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-09-11T08:45:30.488 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:45:30.509 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T08:45:30.509 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:45:30.531 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:45:30.576 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T08:45:30.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:45:30.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:45:30.722 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T08:45:30.732 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T08:45:30.732 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:45:30.758 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:45:30.783 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T08:45:30.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:45:30.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:45:30.823 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-09-11T08:45:30.847 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-09-11T08:45:30.869 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:45:30.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.891 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:45:30.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T08:45:30.933 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T08:45:30.934 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/remote/smithi045 2024-09-11T08:45:30.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:45:30.970 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900665/remote/smithi086 2024-09-11T08:45:30.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:45:31.002 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T08:45:31.002 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:45:31.006 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:45:31.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T08:45:31.064 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T08:45:31.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T08:45:31.076 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T08:45:31.076 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:45:31.078 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:45:31.091 INFO:teuthology.orchestra.run.smithi045.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:45 /home/ubuntu/cephtest 2024-09-11T08:45:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:45 /home/ubuntu/cephtest 2024-09-11T08:45:31.108 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T08:45:31.117 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T08:45:31.137 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/journal} duration: 571.62904047966 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-11T08:45:31.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:45:31.222 INFO:teuthology.run:pass